notebooks/4.1 Exploratory data analysis and working with texts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploratory data analysis and working with texts\n",
    "\n",
    "In this notebook, we learn about:\n",
    "1. descriptive statistics to explore data;\n",
    "2. working with texts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1: descriptive statistics\n",
    "\n",
    "*\"The goal of exploratory data analysis is to develop an understanding of your data. EDA is fundamentally a creative process. And like most creative processes, the key to asking quality questions is to generate a large quantity of questions.\"* \n",
    "\n",
    "Key questions:\n",
    "* Which kind of variation occurs within variables?\n",
    "* Which kind of co-variation occurs between variables?\n",
    "\n",
    "https://r4ds.had.co.nz/exploratory-data-analysis.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "\n",
    "import os, codecs\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import the dataset\n",
    "Let us import the Venetian apprenticeship contracts dataset in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "root_folder = \"../data/apprenticeship_venice/\"\n",
    "df_contracts = pd.read_csv(codecs.open(os.path.join(root_folder,\"professions_data.csv\"), encoding=\"utf8\"), sep=\";\")\n",
    "df_professions = pd.read_csv(codecs.open(os.path.join(root_folder,\"professions_classification.csv\"), encoding=\"utf8\"), sep=\",\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take another look to the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 9653 entries, 0 to 9652\n",
      "Data columns (total 47 columns):\n",
      "page_title                  9653 non-null object\n",
      "register                    9653 non-null object\n",
      "annual_salary               7870 non-null float64\n",
      "a_profession                9653 non-null object\n",
      "profession_code_strict      9618 non-null object\n",
      "profession_code_gen         9614 non-null object\n",
      "profession_cat              9597 non-null object\n",
      "corporation                 9350 non-null object\n",
      "keep_profession_a           9653 non-null int64\n",
      "complete_profession_a       9653 non-null int64\n",
      "enrolmentY                  9628 non-null float64\n",
      "enrolmentM                  9631 non-null float64\n",
      "startY                      9533 non-null float64\n",
      "startM                      9539 non-null float64\n",
      "length                      9645 non-null float64\n",
      "has_fled                    9653 non-null int64\n",
      "m_profession                9535 non-null object\n",
      "m_profession_code_strict    9508 non-null object\n",
      "m_profession_code_gen       9506 non-null object\n",
      "m_profession_cat            9489 non-null object\n",
      "m_corporation               9276 non-null object\n",
      "keep_profession_m           9653 non-null int64\n",
      "complete_profession_m       9653 non-null int64\n",
      "m_gender                    9554 non-null float64\n",
      "m_name                      9623 non-null object\n",
      "m_surname                   6960 non-null object\n",
      "m_patronimic                2620 non-null object\n",
      "m_atelier                   1434 non-null object\n",
      "m_coords                    9639 non-null object\n",
      "a_name                      9653 non-null object\n",
      "a_age                       9303 non-null float64\n",
      "a_gender                    9522 non-null float64\n",
      "a_geo_origins               7149 non-null object\n",
      "a_geo_origins_std           4636 non-null object\n",
      "a_coords                    9610 non-null object\n",
      "a_quondam                   7848 non-null float64\n",
      "accommodation_master        9653 non-null int64\n",
      "personal_care_master        9653 non-null int64\n",
      "clothes_master              9653 non-null int64\n",
      "generic_expenses_master     9653 non-null int64\n",
      "salary_in_kind_master       9653 non-null int64\n",
      "pledge_goods_master         9653 non-null int64\n",
      "pledge_money_master         9653 non-null int64\n",
      "salary_master               9653 non-null int64\n",
      "female_guarantor            9653 non-null int64\n",
      "period_cat                  7891 non-null float64\n",
      "incremental_salary          9653 non-null int64\n",
      "dtypes: float64(11), int64(15), object(21)\n",
      "memory usage: 3.5+ MB\n"
     ]
    }
   ],
   "source": [
    "df_contracts.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>page_title</th>\n",
       "      <th>register</th>\n",
       "      <th>annual_salary</th>\n",
       "      <th>a_profession</th>\n",
       "      <th>profession_code_strict</th>\n",
       "      <th>profession_code_gen</th>\n",
       "      <th>profession_cat</th>\n",
       "      <th>corporation</th>\n",
       "      <th>keep_profession_a</th>\n",
       "      <th>complete_profession_a</th>\n",
       "      <th>...</th>\n",
       "      <th>personal_care_master</th>\n",
       "      <th>clothes_master</th>\n",
       "      <th>generic_expenses_master</th>\n",
       "      <th>salary_in_kind_master</th>\n",
       "      <th>pledge_goods_master</th>\n",
       "      <th>pledge_money_master</th>\n",
       "      <th>salary_master</th>\n",
       "      <th>female_guarantor</th>\n",
       "      <th>period_cat</th>\n",
       "      <th>incremental_salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Carlo Della sosta (Orese) 1592-08-03</td>\n",
       "      <td>asv, giustizia vecchia, accordi dei garzoni, 1...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>orese</td>\n",
       "      <td>orese</td>\n",
       "      <td>orefice</td>\n",
       "      <td>orefice</td>\n",
       "      <td>Oresi</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Antonio quondam Andrea (squerariol) 1583-01-09</td>\n",
       "      <td>asv, giustizia vecchia, accordi dei garzoni, 1...</td>\n",
       "      <td>12.5</td>\n",
       "      <td>squerariol</td>\n",
       "      <td>squerariol</td>\n",
       "      <td>lavori allo squero</td>\n",
       "      <td>lavori allo squero</td>\n",
       "      <td>Squerarioli</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Cristofollo di Zuane (batioro in carta) 1591-0...</td>\n",
       "      <td>asv, giustizia vecchia, accordi dei garzoni, 1...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>batioro</td>\n",
       "      <td>batioro</td>\n",
       "      <td>battioro</td>\n",
       "      <td>fabbricatore di foglie/fili/cordelle d'oro o a...</td>\n",
       "      <td>Battioro</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Illeggibile (marzer) 1584-06-21</td>\n",
       "      <td>asv, giustizia vecchia, accordi dei garzoni, 1...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>marzer</td>\n",
       "      <td>marzer</td>\n",
       "      <td>marzer</td>\n",
       "      <td>merciaio</td>\n",
       "      <td>Merzeri</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Domenico Morebetti (spechier) 1664-09-13</td>\n",
       "      <td>asv, giustizia vecchia, accordi dei garzoni, 1...</td>\n",
       "      <td>7.0</td>\n",
       "      <td>marzer</td>\n",
       "      <td>marzer</td>\n",
       "      <td>marzer</td>\n",
       "      <td>merciaio</td>\n",
       "      <td>Merzeri</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 47 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          page_title  \\\n",
       "0               Carlo Della sosta (Orese) 1592-08-03   \n",
       "1     Antonio quondam Andrea (squerariol) 1583-01-09   \n",
       "2  Cristofollo di Zuane (batioro in carta) 1591-0...   \n",
       "3                    Illeggibile (marzer) 1584-06-21   \n",
       "4           Domenico Morebetti (spechier) 1664-09-13   \n",
       "\n",
       "                                            register  annual_salary  \\\n",
       "0  asv, giustizia vecchia, accordi dei garzoni, 1...            NaN   \n",
       "1  asv, giustizia vecchia, accordi dei garzoni, 1...           12.5   \n",
       "2  asv, giustizia vecchia, accordi dei garzoni, 1...            NaN   \n",
       "3  asv, giustizia vecchia, accordi dei garzoni, 1...            NaN   \n",
       "4  asv, giustizia vecchia, accordi dei garzoni, 1...            7.0   \n",
       "\n",
       "  a_profession profession_code_strict profession_code_gen  \\\n",
       "0        orese                  orese             orefice   \n",
       "1   squerariol             squerariol  lavori allo squero   \n",
       "2      batioro                batioro            battioro   \n",
       "3       marzer                 marzer              marzer   \n",
       "4       marzer                 marzer              marzer   \n",
       "\n",
       "                                      profession_cat  corporation  \\\n",
       "0                                            orefice        Oresi   \n",
       "1                                 lavori allo squero  Squerarioli   \n",
       "2  fabbricatore di foglie/fili/cordelle d'oro o a...     Battioro   \n",
       "3                                           merciaio      Merzeri   \n",
       "4                                           merciaio      Merzeri   \n",
       "\n",
       "   keep_profession_a  complete_profession_a  ...  personal_care_master  \\\n",
       "0                  1                      1  ...                     1   \n",
       "1                  1                      1  ...                     0   \n",
       "2                  1                      1  ...                     0   \n",
       "3                  1                      1  ...                     0   \n",
       "4                  1                      1  ...                     0   \n",
       "\n",
       "   clothes_master  generic_expenses_master  salary_in_kind_master  \\\n",
       "0               1                        1                      0   \n",
       "1               0                        1                      0   \n",
       "2               0                        0                      0   \n",
       "3               0                        0                      0   \n",
       "4               0                        1                      0   \n",
       "\n",
       "   pledge_goods_master  pledge_money_master salary_master female_guarantor  \\\n",
       "0                    0                    0             0                0   \n",
       "1                    0                    0             1                0   \n",
       "2                    0                    0             0                0   \n",
       "3                    0                    0             0                0   \n",
       "4                    0                    0             1                0   \n",
       "\n",
       "  period_cat incremental_salary  \n",
       "0        NaN                  0  \n",
       "1        1.0                  0  \n",
       "2        NaN                  0  \n",
       "3        NaN                  0  \n",
       "4        1.0                  0  \n",
       "\n",
       "[5 rows x 47 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_contracts.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['page_title', 'register', 'annual_salary', 'a_profession',\n",
       "       'profession_code_strict', 'profession_code_gen', 'profession_cat',\n",
       "       'corporation', 'keep_profession_a', 'complete_profession_a',\n",
       "       'enrolmentY', 'enrolmentM', 'startY', 'startM', 'length', 'has_fled',\n",
       "       'm_profession', 'm_profession_code_strict', 'm_profession_code_gen',\n",
       "       'm_profession_cat', 'm_corporation', 'keep_profession_m',\n",
       "       'complete_profession_m', 'm_gender', 'm_name', 'm_surname',\n",
       "       'm_patronimic', 'm_atelier', 'm_coords', 'a_name', 'a_age', 'a_gender',\n",
       "       'a_geo_origins', 'a_geo_origins_std', 'a_coords', 'a_quondam',\n",
       "       'accommodation_master', 'personal_care_master', 'clothes_master',\n",
       "       'generic_expenses_master', 'salary_in_kind_master',\n",
       "       'pledge_goods_master', 'pledge_money_master', 'salary_master',\n",
       "       'female_guarantor', 'period_cat', 'incremental_salary'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_contracts.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every row represents an apprenticeship contract. Contracts were registered both at the guild's and at a public office. This is a sample of contracts from a much larger set of records.\n",
    "\n",
    "Some of the variables we will work with are:\n",
    "* `annual_salary`: the annual salary paid to the apprencice, if any (in Venetian ducats).\n",
    "* `a_profession` to `corporation`: increasingly generic classifications for the apprentice's stated profession.\n",
    "* `startY` and `enrolmentY`: contract start and registration year respectively.\n",
    "* `length`: of the contract, in years.\n",
    "* `m_gender` and `a_gender`: of master and apprentice respectively.\n",
    "* `a_age`: age of the apprentice at entry, in years.\n",
    "* `female_guarantor`: if at least one of the contract's guarantors was female, boolean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Trascrizione</th>\n",
       "      <th>Standard</th>\n",
       "      <th>Gruppo 0</th>\n",
       "      <th>Gruppo 1</th>\n",
       "      <th>Gruppo 2</th>\n",
       "      <th>Gruppo 3</th>\n",
       "      <th>Gruppo 4</th>\n",
       "      <th>Corporazione</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>al negotio del libraro</td>\n",
       "      <td>librer</td>\n",
       "      <td>libraio</td>\n",
       "      <td>librai - diverse specializzazioni</td>\n",
       "      <td>stampa</td>\n",
       "      <td>altre lavorazioni manifatturiere</td>\n",
       "      <td>beni</td>\n",
       "      <td>libreri, stampatori e ligadori</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>arte de far arpicordi</td>\n",
       "      <td>arte de far arpicordi</td>\n",
       "      <td>fabbricatore di arpicordi</td>\n",
       "      <td>fabbricatore di strumenti musicali</td>\n",
       "      <td>musica</td>\n",
       "      <td>altri servizi</td>\n",
       "      <td>servizi</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>arte de' colori</td>\n",
       "      <td>arte dei colori</td>\n",
       "      <td>fabbricazione/vendita di colori</td>\n",
       "      <td>colori</td>\n",
       "      <td>colori</td>\n",
       "      <td>decorazioni e mestieri dell'arte</td>\n",
       "      <td>beni</td>\n",
       "      <td>spezieri</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Trascrizione               Standard  \\\n",
       "0  al negotio del libraro                 librer   \n",
       "1   arte de far arpicordi  arte de far arpicordi   \n",
       "2         arte de' colori        arte dei colori   \n",
       "\n",
       "                          Gruppo 0                            Gruppo 1  \\\n",
       "0                          libraio   librai - diverse specializzazioni   \n",
       "1        fabbricatore di arpicordi  fabbricatore di strumenti musicali   \n",
       "2  fabbricazione/vendita di colori                              colori   \n",
       "\n",
       "  Gruppo 2                          Gruppo 3 Gruppo 4  \\\n",
       "0   stampa  altre lavorazioni manifatturiere     beni   \n",
       "1   musica                     altri servizi  servizi   \n",
       "2   colori  decorazioni e mestieri dell'arte     beni   \n",
       "\n",
       "                     Corporazione  \n",
       "0  libreri, stampatori e ligadori  \n",
       "1                             NaN  \n",
       "2                        spezieri  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_professions.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The professions data frame contains a classification system for each profession as found in the records (transcription, first column). The last column is the guild (or corporation) which governed the given profession. This work was performed manually by historians. We don't use it here as the classifications we need are already part of the main dataframe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Questions\n",
    "\n",
    "* Plot the distribution (histogram) of the apprentices' age, contract length, annual salary and start year.\n",
    "* Calculate the proportion of female apprentices and masters, and of contracts with a female guarantor.\n",
    "* How likely it is for a female apprentice to have a female master? And for a male apprentice?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11cf359b0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAATSUlEQVR4nO3df4xd9Znf8fenkGRVJgVTshY17ppU3lXZoLIwAqR0V2OlC4a066RtKhAiJj/krQRSok2lOBttQZuN5G2TrBptytZZrJBuNrNUSRQLSFnXwkX5gw2YOhjHYfESN7VxbW1gTZxEaUmf/nHPWNfDjGc8c+feGX/fL+nqnvOc7zn3uWfGn3vm3HOvU1VIktrwt0bdgCRpeAx9SWqIoS9JDTH0Jakhhr4kNeTCUTdwNpdddlmtW7duQev+6Ec/4qKLLhpsQ0tkpfRqn4O3Unq1z8Fbyl737t3711X1lhkXVtWyvV133XW1UI8//viC1x22ldKrfQ7eSunVPgdvKXsFnq5ZctXTO5LUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1JBl/TUMg7Ru6yOnpw9ve+cIO5Gk0fFIX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWrInKGfZG2Sx5McTHIgyYe6+n1JjibZ191u7VvnY0kOJXk+yc199Y1d7VCSrUvzlCRJs5nPf4z+GvCRqnomyZuBvUl2dcv+oKo+1T84yVXAbcAvA38P+G9JfrFb/Dng14EjwFNJdlbVdwbxRCRJc5sz9KvqGHCsm/5hkoPAmrOssgmYrKqfAt9Lcgi4vlt2qKpeBEgy2Y019CVpSFJV8x+crAOeAN4G/BZwF/Aq8DS9vwZeSfKHwJNV9SfdOg8A3+g2sbGqPtjV7wRuqKp7pj3GFmALwOrVq6+bnJxc0BM7deoUY2Njp+f3Hz15evrqNRcvaJtLZXqvy5V9Dt5K6dU+B28pe92wYcPeqhqfadl8Tu8AkGQM+Arw4ap6Ncn9wCeA6u4/DbwfyAyrFzO/f/C6V5yq2g5sBxgfH6+JiYn5tniGPXv20L/uXVsfOT19+I6FbXOpTO91ubLPwVspvdrn4I2q13mFfpI30Av8L1XVVwGq6njf8s8DD3ezR4C1fatfAbzUTc9WlyQNwXyu3gnwAHCwqj7TV7+8b9i7gee66Z3AbUnelORKYD3wLeApYH2SK5O8kd6bvTsH8zQkSfMxnyP9twN3AvuT7Otqvw3cnuQaeqdoDgO/CVBVB5I8RO8N2teAu6vqZwBJ7gEeAy4AdlTVgQE+F0nSHOZz9c43mfk8/aNnWeeTwCdnqD96tvUkSUvLT+RKUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhc/7H6OejdVsfOT19eNs7R9iJJA2XR/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDVkztBPsjbJ40kOJjmQ5ENd/dIku5K80N2v6upJ8tkkh5I8m+Tavm1t7sa/kGTz0j0tSdJM5nOk/xrwkar6h8CNwN1JrgK2Aruraj2wu5sHuAVY3922APdD70UCuBe4AbgeuHfqhUKSNBxzhn5VHauqZ7rpHwIHgTXAJuDBbtiDwLu66U3AF6vnSeCSJJcDNwO7qurlqnoF2AVsHOizkSSdVapq/oOTdcATwNuA71fVJX3LXqmqVUkeBrZV1Te7+m7go8AE8HNV9Xtd/XeAn1TVp6Y9xhZ6fyGwevXq6yYnJxf0xE6dOsXY2Njp+f1HT8447uo1Fy9o+4M0vdflyj4Hb6X0ap+Dt5S9btiwYW9Vjc+0bN7fvZNkDPgK8OGqejXJrENnqNVZ6mcWqrYD2wHGx8drYmJivi2eYc+ePfSve1ff9+30O3zHwrY/SNN7Xa7sc/BWSq/2OXij6nVeV+8keQO9wP9SVX21Kx/vTtvQ3Z/o6keAtX2rXwG8dJa6JGlI5nP1ToAHgINV9Zm+RTuBqStwNgNf76u/t7uK50bgZFUdAx4DbkqyqnsD96auJkkakvmc3nk7cCewP8m+rvbbwDbgoSQfAL4PvKdb9ihwK3AI+DHwPoCqejnJJ4CnunG/W1UvD+RZSJLmZc7Q796Qne0E/jtmGF/A3bNsawew41walCQNjp/IlaSGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkPmDP0kO5KcSPJcX+2+JEeT7Otut/Yt+1iSQ0meT3JzX31jVzuUZOvgn4okaS7zOdL/ArBxhvofVNU13e1RgCRXAbcBv9yt8x+TXJDkAuBzwC3AVcDt3VhJ0hBdONeAqnoiybp5bm8TMFlVPwW+l+QQcH237FBVvQiQZLIb+51z7liStGCpqrkH9UL/4ap6Wzd/H3AX8CrwNPCRqnolyR8CT1bVn3TjHgC+0W1mY1V9sKvfCdxQVffM8FhbgC0Aq1evvm5ycnJBT+zUqVOMjY2dnt9/9OSM465ec/GCtj9I03tdruxz8FZKr/Y5eEvZ64YNG/ZW1fhMy+Y80p/F/cAngOruPw28H8gMY4uZTyPN+GpTVduB7QDj4+M1MTGxoAb37NlD/7p3bX1kxnGH71jY9gdpeq/LlX0O3krp1T4Hb1S9Lij0q+r41HSSzwMPd7NHgLV9Q68AXuqmZ6tLkoZkQZdsJrm8b/bdwNSVPTuB25K8KcmVwHrgW8BTwPokVyZ5I703e3cuvG1J0kLMeaSf5MvABHBZkiPAvcBEkmvonaI5DPwmQFUdSPIQvTdoXwPurqqfddu5B3gMuADYUVUHBv5sJElnNZ+rd26fofzAWcZ/EvjkDPVHgUfPqTtJ0kD5iVxJaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqyIL+Y/Tzybqtj5yePrztnSPsRJKWnkf6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUkDlDP8mOJCeSPNdXuzTJriQvdPerunqSfDbJoSTPJrm2b53N3fgXkmxemqcjSTqb+RzpfwHYOK22FdhdVeuB3d08wC3A+u62Bbgfei8SwL3ADcD1wL1TLxSSpOGZM/Sr6gng5WnlTcCD3fSDwLv66l+snieBS5JcDtwM7Kqql6vqFWAXr38hkSQtsVTV3IOSdcDDVfW2bv5vquqSvuWvVNWqJA8D26rqm119N/BRYAL4uar6va7+O8BPqupTMzzWFnp/JbB69errJicnF/TETp06xdjY2On5/UdPzrnO1WsuXtBjLdb0Xpcr+xy8ldKrfQ7eUva6YcOGvVU1PtOyQX/LZmao1Vnqry9WbQe2A4yPj9fExMSCGtmzZw/9697V922aszl8x8Iea7Gm97pc2efgrZRe7XPwRtXrQq/eOd6dtqG7P9HVjwBr+8ZdAbx0lrokaYgWGvo7gakrcDYDX++rv7e7iudG4GRVHQMeA25Ksqp7A/emriZJGqI5T+8k+TK9c/KXJTlC7yqcbcBDST4AfB94Tzf8UeBW4BDwY+B9AFX1cpJPAE914363qqa/OSxJWmJzhn5V3T7LonfMMLaAu2fZzg5gxzl1J0kaKD+RK0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYsKvSTHE6yP8m+JE93tUuT7EryQne/qqsnyWeTHErybJJrB/EEJEnzN4gj/Q1VdU1VjXfzW4HdVbUe2N3NA9wCrO9uW4D7B/DYkqRzsBSndzYBD3bTDwLv6qt/sXqeBC5JcvkSPL4kaRapqoWvnHwPeAUo4D9V1fYkf1NVl/SNeaWqViV5GNhWVd/s6ruBj1bV09O2uYXeXwKsXr36usnJyQX1durUKcbGxk7P7z96cs51rl5z8YIea7Gm97pc2efgrZRe7XPwlrLXDRs27O07+3KGCxe57bdX1UtJfh7YleS7ZxmbGWqve8Wpqu3AdoDx8fGamJhYUGN79uyhf927tj4y5zqH71jYYy3W9F6XK/scvJXSq30O3qh6XdTpnap6qbs/AXwNuB44PnXaprs/0Q0/AqztW/0K4KXFPL4k6dwsOPSTXJTkzVPTwE3Ac8BOYHM3bDPw9W56J/De7iqeG4GTVXVswZ1Lks7ZYk7vrAa+lmRqO39aVf81yVPAQ0k+AHwfeE83/lHgVuAQ8GPgfYt4bEnSAiw49KvqReAfzVD/AfCOGeoF3L3Qx5MkLZ6fyJWkhiz26p3zyrq+K3wOb3vnCDuRpKXhkb4kNeS8PtJfN49r8yWpJR7pS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpyXn84azH8SgZJ5yOP9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDvGRzHrx8U9L5wiN9SWqIoS9JDTH0Jakhhr4kNcQ3chfBN3glrTSG/hLzhUHScmLon6P+EJeklcbQH5H+F48vbLxohJ1IasnQQz/JRuA/ABcAf1xV24bdw1JYitM4nhqSNGhDDf0kFwCfA34dOAI8lWRnVX1nmH20yBcQSTD8I/3rgUNV9SJAkklgE3Behf5s5/1nq+8/epK7umXnGsjTt7kUgT71GB+5+jUmBrAd8IVHGpVU1fAeLPmXwMaq+mA3fydwQ1Xd0zdmC7Clm/0l4PkFPtxlwF8vot1hWim92ufgrZRe7XPwlrLXX6iqt8y0YNhH+pmhdsarTlVtB7Yv+oGSp6tqfLHbGYaV0qt9Dt5K6dU+B29UvQ77E7lHgLV981cALw25B0lq1rBD/ylgfZIrk7wRuA3YOeQeJKlZQz29U1WvJbkHeIzeJZs7qurAEj3cok8RDdFK6dU+B2+l9GqfgzeSXof6Rq4kabT8lk1JaoihL0kNOS9DP8nGJM8nOZRk66j7mZJkbZLHkxxMciDJh7r6fUmOJtnX3W4dda8ASQ4n2d/19HRXuzTJriQvdPerRtzjL/Xtt31JXk3y4eWwT5PsSHIiyXN9tRn3X3o+2/3OPpvk2mXQ679P8t2un68luaSrr0vyk759+0cj7nPWn3WSj3X79PkkN4+4zz/r6/Fwkn1dfbj7s6rOqxu9N4j/Cngr8Ebg28BVo+6r6+1y4Npu+s3AXwJXAfcB/2bU/c3Q72Hgsmm1fwds7aa3Ar8/6j6n/ez/N/ALy2GfAr8GXAs8N9f+A24FvkHvsyw3An+xDHq9Cbiwm/79vl7X9Y9bBn3O+LPu/m19G3gTcGWXCxeMqs9pyz8N/NtR7M/z8Uj/9Fc9VNX/Aaa+6mHkqupYVT3TTf8QOAisGW1X52wT8GA3/SDwrhH2Mt07gL+qqv856kYAquoJ4OVp5dn23ybgi9XzJHBJksuH0+nMvVbVn1fVa93sk/Q+VzNSs+zT2WwCJqvqp1X1PeAQvXxYcmfrM0mAfwV8eRi9THc+hv4a4H/1zR9hGQZrknXArwB/0ZXu6f6M3jHqUyZ9CvjzJHu7r8cAWF1Vx6D3Igb8/Mi6e73bOPMf0nLcp7Ptv+X+e/t+en+JTLkyyf9I8t+T/Oqomuoz0896ue7TXwWOV9ULfbWh7c/zMfTn/KqHUUsyBnwF+HBVvQrcD/wD4BrgGL0//ZaDt1fVtcAtwN1Jfm3UDc2m+7DfbwD/pSst1306m2X7e5vk48BrwJe60jHg71fVrwC/Bfxpkr8zqv6Y/We9XPfp7Zx5cDLU/Xk+hv6y/qqHJG+gF/hfqqqvAlTV8ar6WVX9P+DzDOlP0LlU1Uvd/Qnga/T6Oj512qG7PzG6Ds9wC/BMVR2H5btPmX3/Lcvf2ySbgX8K3FHdCejudMkPuum99M6V/+KoejzLz3rZ7dMkFwL/HPizqdqw9+f5GPrL9qseunN5DwAHq+ozffX+c7fvBp6bvu6wJbkoyZunpum9qfccvX25uRu2Gfj6aDp8nTOOnpbjPu3Mtv92Au/truK5ETg5dRpoVNL7D48+CvxGVf24r/6W9P5vDJK8FVgPvDiaLs/6s94J3JbkTUmupNfnt4bd3zT/BPhuVR2ZKgx9fw7rHeNh3uhdCfGX9F4xPz7qfvr6+sf0/rx8FtjX3W4F/jOwv6vvBC5fBr2+ld6VD98GDkztR+DvAruBF7r7S5dBr38b+AFwcV9t5PuU3ovQMeD/0jvq/MBs+4/eqYjPdb+z+4HxZdDrIXrnxKd+V/+oG/svut+JbwPPAP9sxH3O+rMGPt7t0+eBW0bZZ1f/AvCvp40d6v70axgkqSHn4+kdSdIsDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUkP8PphszS0QeCmwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_contracts.annual_salary.hist(bins=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x10bdaf588>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD4CAYAAADo30HgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAV3klEQVR4nO3df5Bd5X3f8fe3yGDsdSQB8ZZKmgo3GrcOahLYARK3nlWUYgEei3ZMBg9jC4eMxlNISa1MketpyKT1RG5KHLtNnVECY7lhvDjELio/amtkth7/IWKLYgTGjhai2gJF1EWWs4bUUfrtH/dZ+z7L3R/3nnvv7sL7NbOz5z7nOed877nn3s+eH/dsZCaSJM34W0tdgCRpeTEYJEkVg0GSVDEYJEkVg0GSVFm11AXM54ILLsiNGzd2Nc33v/99Xv/61w+moD6wvmasrxnra2al1Hf48OHvZOaP9zyjzFy2P5deeml26+GHH+56mmGyvmasrxnra2al1Ad8NRt89nooSZJUMRgkSRWDQZJUMRgkSRWDQZJUMRgkSRWDQZJUMRgkSRWDQZJUWda3xHil2bj7AXZtPsONux9YVP9je64ZcEWS9HLuMUiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKmyYDBExF0R8XxEPNHW9tsR8Y2IeDwiPhcRa9rGfTAipiLimxHx9rb2baVtKiJ29/+pSJL6YTF7DJ8Ets1qOwBcnJn/EPgz4IMAEfEW4HrgJ8s0/zkizoqIs4DfA64C3gK8u/SVJC0zCwZDZn4JeGFW2xcy80x5eAhYX4a3AxOZ+X8z88+BKeCy8jOVmc9k5g+AidJXkrTMRGYu3CliI3B/Zl7cYdx/A+7JzD+KiP8EHMrMPyrj7gQeKl23ZeYvl/b3AJdn5i0d5rcT2AkwOjp66cTERFdPaHp6mpGRka6mGZYjz55m9Fw4+dLi+m9et3qwBXWwnNcfWF9T1tfMSqlvy5YthzNzrNf5NLrtdkR8CDgD3D3T1KFb0nnPpGMiZeZeYC/A2NhYjo+Pd1XT5OQk3U4zLDeW227fcWRxq/3YDeODLaiD5bz+wPqasr5mXi319RwMEbEDeAewNX+023Ec2NDWbT3wXBmeq12StIz0dLlqRGwDbgPemZkvto3aD1wfEedExEXAJuBPga8AmyLioog4m9YJ6v3NSpckDcKCewwR8WlgHLggIo4Dt9O6Cukc4EBEQOu8wvsz88mI+AzwdVqHmG7OzL8p87kF+DxwFnBXZj45gOcjSWpowWDIzHd3aL5znv4fBj7cof1B4MGuqpMkDZ3ffJYkVRpdlaTlZ+PuB7rqf2zPNQOqRNJK5R6DJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKmyYDBExF0R8XxEPNHWdl5EHIiIo+X32tIeEfHxiJiKiMcj4pK2aXaU/kcjYsdgno4kqanF7DF8Etg2q203cDAzNwEHy2OAq4BN5Wcn8AloBQlwO3A5cBlw+0yYSJKWlwWDITO/BLwwq3k7sK8M7wOubWv/VLYcAtZExIXA24EDmflCZp4CDvDysJEkLQORmQt3itgI3J+ZF5fH383MNW3jT2Xm2oi4H9iTmV8u7QeB24Bx4LWZ+e9K+78BXsrM/9BhWTtp7W0wOjp66cTERFdPaHp6mpGRka6mGZYjz55m9Fw4+dLi+m9et7qnZXRj9jKW8/oD62vK+ppZKfVt2bLlcGaO9TqfVf0sCogObTlP+8sbM/cCewHGxsZyfHy8qwImJyfpdpphuXH3A+zafIY7jixutR+7YbynZXRj9jKW8/oD62vK+pp5tdTX61VJJ8shIsrv50v7cWBDW7/1wHPztEuSlpleg2E/MHNl0Q7gvrb295ark64ATmfmCeDzwJURsbacdL6ytEmSlpkFj2lExKdpnSO4ICKO07q6aA/wmYi4CfgWcF3p/iBwNTAFvAi8DyAzX4iIfwt8pfT7zcycfUJbkrQMLBgMmfnuOUZt7dA3gZvnmM9dwF1dVSdJGjq/+SxJqvT7qiStMBtnXcW0a/OZea9sOrbnmkGXJGmJuccgSaoYDJKkisEgSap4jqGB2cfnJemVwD0GSVLFYJAkVQwGSVLFYJAkVQwGSVLFYJAkVQwGSVLFYJAkVQwGSVLFYJAkVQwGSVLFYJAkVQwGSVLFYJAkVQwGSVLFYJAkVQwGSVKlUTBExL+MiCcj4omI+HREvDYiLoqIRyLiaETcExFnl77nlMdTZfzGfjwBSVJ/9RwMEbEO+BfAWGZeDJwFXA98BPhoZm4CTgE3lUluAk5l5k8AHy39JEnLTNNDSauAcyNiFfA64ATw88C9Zfw+4NoyvL08pozfGhHRcPmSpD6LzOx94ohbgQ8DLwFfAG4FDpW9AiJiA/BQZl4cEU8A2zLzeBn3NHB5Zn5n1jx3AjsBRkdHL52YmOiqpunpaUZGRnp+Tt048uzprqcZPRdOvrS4vpvXre56/r3U1G6h+nqpqZ+G+fr2wvqasb5mZurbsmXL4cwc63U+q3qdMCLW0toLuAj4LvDHwFUdus4kT6e9g5elUmbuBfYCjI2N5fj4eFd1TU5O0u00vbpx9wNdT7Nr8xnuOLK41X7shvGu599LTe0Wqq+XmvppmK9vL6yvGetrpl/1NTmU9AvAn2fm/87MvwY+C/wcsKYcWgJYDzxXho8DGwDK+NXACw2WL0kagCbB8C3gioh4XTlXsBX4OvAw8K7SZwdwXxneXx5Txn8xmxzHkiQNRM/BkJmP0DqJ/ChwpMxrL3Ab8IGImALOB+4sk9wJnF/aPwDsblC3JGlAej7HAJCZtwO3z2p+BrisQ9+/Aq5rsjwtvY1dnsM4tueaAVUiaVD85rMkqWIwSJIqBoMkqWIwSJIqBoMkqWIwSJIqBoMkqWIwSJIqBoMkqWIwSJIqBoMkqWIwSJIqjW6ip8Hq9oZ1ktQP7jFIkioGgySpYjBIkioGgySpYjBIkioGgySpYjBIkioGgySpYjBIkioGgySp0igYImJNRNwbEd+IiKci4mcj4ryIOBARR8vvtaVvRMTHI2IqIh6PiEv68xQkSf3UdI/hY8B/z8y/D/wU8BSwGziYmZuAg+UxwFXApvKzE/hEw2VLkgag52CIiB8D3gbcCZCZP8jM7wLbgX2l2z7g2jK8HfhUthwC1kTEhT1XLkkaiMjM3iaM+GlgL/B1WnsLh4FbgWczc01bv1OZuTYi7gf2ZOaXS/tB4LbM/Oqs+e6ktUfB6OjopRMTE13VNT09zcjISE/PqVtHnj3d9TSj58LJlwZQTJ/0u77N61b3b2YM9/XthfU1Y33NzNS3ZcuWw5k51ut8mtx2exVwCfArmflIRHyMHx026iQ6tL0slTJzL63AYWxsLMfHx7sqanJykm6n6dWNPdwWe9fmM9xxZPne7bzf9R27Ybxv84Lhvr69sL5mrK+ZftXX5BzDceB4Zj5SHt9LKyhOzhwiKr+fb+u/oW369cBzDZYvSRqAnoMhM/8C+HZEvLk0baV1WGk/sKO07QDuK8P7gfeWq5OuAE5n5olely9JGoymxwx+Bbg7Is4GngHeRytsPhMRNwHfAq4rfR8ErgamgBdLX0nSMtMoGDLzMaDTCY6tHfomcHOT5UmSBs9vPkuSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKgaDJKliMEiSKo2DISLOioj/GRH3l8cXRcQjEXE0Iu6JiLNL+znl8VQZv7HpsiVJ/dePPYZbgafaHn8E+GhmbgJOATeV9puAU5n5E8BHSz9J0jLTKBgiYj1wDfCH5XEAPw/cW7rsA64tw9vLY8r4raW/JGkZiczsfeKIe4HfAt4A/BpwI3Co7BUQERuAhzLz4oh4AtiWmcfLuKeByzPzO7PmuRPYCTA6OnrpxMREVzVNT08zMjLS83PqxpFnT3c9zei5cPKlARTTJ/2ub/O61f2bGcN9fXthfc1YXzMz9W3ZsuVwZo71Op9VvU4YEe8Ans/MwxExPtPcoWsuYtyPGjL3AnsBxsbGcnx8fHaXeU1OTtLtNL26cfcDXU+za/MZ7jjS82ofuH7Xd+yG8b7NC4b7+vbC+pqxvmb6VV+TT4C3Au+MiKuB1wI/BvwusCYiVmXmGWA98FzpfxzYAByPiFXAauCFBsuXJA1Az+cYMvODmbk+MzcC1wNfzMwbgIeBd5VuO4D7yvD+8pgy/ovZ5DiWJGkgBvE9htuAD0TEFHA+cGdpvxM4v7R/ANg9gGVLkhrqy8HkzJwEJsvwM8BlHfr8FXBdP5YnSRocv/ksSaoYDJKkyvK9blKvCBu7vKT32J5rBlSJpMVyj0GSVDEYJEkVg0GSVDEYJEkVg0GSVDEYJEkVg0GSVDEYJEkVg0GSVDEYJEkVg0GSVPFeSW26va+PJL0SuccgSaoYDJKkisEgSaoYDJKkisEgSap4VZK0gG6uVtu1+QzjgytFGgqDQStaL5cY++9Dpfl5KEmSVOk5GCJiQ0Q8HBFPRcSTEXFraT8vIg5ExNHye21pj4j4eERMRcTjEXFJv56EJKl/muwxnAF2ZeY/AK4Abo6ItwC7gYOZuQk4WB4DXAVsKj87gU80WLYkaUB6DobMPJGZj5bhvwSeAtYB24F9pds+4NoyvB34VLYcAtZExIU9Vy5JGoi+nGOIiI3AzwCPAKOZeQJa4QG8sXRbB3y7bbLjpU2StIxEZjabQcQI8D+AD2fmZyPiu5m5pm38qcxcGxEPAL+VmV8u7QeBf5WZh2fNbyetQ02Mjo5eOjEx0VU909PTjIyM9PRcjjx7uqfpujF6Lpx8aeCL6dlS17d53ep5x89+fXt5zRZaxmzdLGP0XHjjed3Nf5iavD+Gwfqamalvy5YthzNzrNf5NLpcNSJeA/wJcHdmfrY0n4yICzPzRDlU9HxpPw5saJt8PfDc7Hlm5l5gL8DY2FiOj493VdPk5CTdTjPjxiHcXXXX5jPccWT5XiW81PUdu2F83vGzX99eXrOFljFbN8vYtfkMv9jj9jcMTd4fw2B9zfSrviZXJQVwJ/BUZv5O26j9wI4yvAO4r639veXqpCuA0zOHnCRJy0eTPw3fCrwHOBIRj5W2fw3sAT4TETcB3wKuK+MeBK4GpoAXgfc1WLYkaUB6DoZyriDmGL21Q/8Ebu51eZKk4Vi+B7v1qrTQLS52bT4zlHNB0quZt8SQJFUMBklSxWCQJFUMBklSxWCQJFUMBklSxWCQJFX8HoO0wnT770z9V6bqlnsMkqSKwSBJqhgMkqSKwSBJqnjyWa863Z68lV5t3GOQJFUMBklSxWCQJFU8xyBp6OY6zzPXP2LyS3rD5R6DJKliMEiSKgaDJKniOQbpFa79eP5cx/Bn6/aY/qC/G+KNA4frFR0MfpFJkrr3ig4GaSXwDxgtN0MPhojYBnwMOAv4w8zcM+waJM3PsFrYK/nw1lCDISLOAn4P+CfAceArEbE/M78+zDqkQfJDden18hqspA/uQRv2HsNlwFRmPgMQERPAdsBgkLSkFhMmiz153+v82y1lUEVmDm9hEe8CtmXmL5fH7wEuz8xb2vrsBHaWh28GvtnlYi4AvtOHcgfF+pqxvmasr5mVUt/fzcwf73Umw95jiA5tVTJl5l5gb88LiPhqZo71Ov2gWV8z1teM9TXzaqlv2F9wOw5saHu8HnhuyDVIkuYx7GD4CrApIi6KiLOB64H9Q65BkjSPoR5KyswzEXEL8Hlal6velZlP9nkxPR+GGhLra8b6mrG+Zl4V9Q315LMkafnzJnqSpIrBIEmqrNhgiIhtEfHNiJiKiN0dxp8TEfeU8Y9ExMYh1rYhIh6OiKci4smIuLVDn/GIOB0Rj5WfXx9WfWX5xyLiSFn2VzuMj4j4eFl/j0fEJUOs7c1t6+WxiPheRPzqrD5DXX8RcVdEPB8RT7S1nRcRByLiaPm9do5pd5Q+RyNixxDr++2I+EZ5/T4XEWvmmHbebWGA9f1GRDzb9hpePce0877XB1jfPW21HYuIx+aYdhjrr+NnysC2wcxccT+0Tlw/DbwJOBv4GvCWWX3+OfD7Zfh64J4h1nchcEkZfgPwZx3qGwfuX8J1eAy4YJ7xVwMP0fruyRXAI0v4Wv8FrS/sLNn6A94GXAI80db274HdZXg38JEO050HPFN+ry3Da4dU35XAqjL8kU71LWZbGGB9vwH82iJe/3nf64Oqb9b4O4BfX8L11/EzZVDb4ErdY/jhrTUy8wfAzK012m0H9pXhe4GtEdHpC3Z9l5knMvPRMvyXwFPAumEsu4+2A5/KlkPAmoi4cAnq2Ao8nZn/awmW/UOZ+SXghVnN7dvYPuDaDpO+HTiQmS9k5ingALBtGPVl5hcy80x5eIjW94aWxBzrbzEW815vbL76yufGLwKf7vdyF2uez5SBbIMrNRjWAd9ue3ycl3/w/rBPeXOcBs4fSnVtyiGsnwEe6TD6ZyPiaxHxUET85FALa33j/AsRcbjchmS2xazjYbieud+QS7n+AEYz8wS03rjAGzv0WS7r8Zdo7QF2stC2MEi3lENdd81xGGQ5rL9/DJzMzKNzjB/q+pv1mTKQbXClBsOCt9ZYZJ+BiogR4E+AX83M780a/SitwyM/BfxH4L8OszbgrZl5CXAVcHNEvG3W+OWw/s4G3gn8cYfRS73+Fms5rMcPAWeAu+fostC2MCifAP4e8NPACVqHa2Zb8vUHvJv59xaGtv4W+EyZc7IObfOuw5UaDIu5tcYP+0TEKmA1ve3K9iQiXkPrBbw7Mz87e3xmfi8zp8vwg8BrIuKCYdWXmc+V388Dn6O1y95uOdy+5Crg0cw8OXvEUq+/4uTM4bXy+/kOfZZ0PZYTje8AbshywHm2RWwLA5GZJzPzbzLz/wF/MMdyl3r9rQL+GXDPXH2Gtf7m+EwZyDa4UoNhMbfW2A/MnH1/F/DFud4Y/VaOSd4JPJWZvzNHn789c84jIi6j9Vr8nyHV9/qIeMPMMK2TlE/M6rYfeG+0XAGcntllHaI5/1JbyvXXpn0b2wHc16HP54ErI2JtOVRyZWkbuGj9U6zbgHdm5otz9FnMtjCo+trPWf3TOZa71LfR+QXgG5l5vNPIYa2/eT5TBrMNDvJM+oDP0l9N68z808CHSttv0noTALyW1iGIKeBPgTcNsbZ/RGtX7XHgsfJzNfB+4P2lzy3Ak7SusjgE/NwQ63tTWe7XSg0z66+9vqD1T5WeBo4AY0N+fV9H64N+dVvbkq0/WgF1AvhrWn+B3UTrnNVB4Gj5fV7pO0brvxPOTPtLZTucAt43xPqmaB1bntkGZ67S+zvAg/NtC0Oq77+UbetxWh9wF86urzx+2Xt9GPWV9k/ObHNtfZdi/c31mTKQbdBbYkiSKiv1UJIkaUAMBklSxWCQJFUMBklSxWCQJFUMBklSxWCQJFX+P1VD/fHnKdbUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_contracts[df_contracts.annual_salary < 20].annual_salary.hist(bins=25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9738941261783901"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_contracts.a_gender.sum()/df_contracts.shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.976276805138299"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_contracts.m_gender.sum()/df_contracts.shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7439024390243902"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_contracts[(df_contracts.a_gender == 0) & (df_contracts.startY < 1600)].m_gender.sum()\n",
    "    /df_contracts[(df_contracts.a_gender == 0) & (df_contracts.startY < 1600)].shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1ab12fd0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAATsElEQVR4nO3dfaxc9X3n8fdnIZAUZ8FZWosAu6aSsxKBNg23wCrb3etEBUP+gGgbCcrykAe5qqBqtXQ3pt0qtDQrbxWaVR6K1hFWyYbNFbtJFgtIkYPqsqnKBkwpxqEUh3hZG2RvamrihE1L+t0/5tiaOHMffH3nznh+75c0mjO/85tzznfOzGfOPefMuakqJElt+AejXgBJ0vIx9CWpIYa+JDXE0Jekhhj6ktSQk0e9AHM588wza/Xq1UOb/ne/+11OO+20oU1/1Ca9Ppj8Gie9Ppj8GkdR3/bt279dVT8+aNxYh/7q1at54oknhjb9bdu2MT09PbTpj9qk1weTX+Ok1weTX+Mo6kvyv2cb5+4dSWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqyFj/IvdEtXrDgyOb9+6N7x3ZvCWNv3m39JOcm+SPkzybZGeSX+3ab0+yN8lT3e3KvufclmRXkueSXN7Xvq5r25Vkw3BKkiTNZiFb+q8Dt1bVk0neDGxPsrUb94mq+nh/5yTnA9cAbwfeCnw1ydu60Z8Bfh7YAzyeZEtVfWMpCpEkzW/e0K+ql4GXu+HvJHkWOHuOp1wFzFTV94FvJdkFXNyN21VVLwAkmen6GvqStExyLP8YPclq4FHgAuDfADcBrwJP0Ptr4JUknwYeq6rPd8+5G/hKN4l1VfXhrv164JKquuWoeawH1gOsWrXqopmZmcXWNq9Dhw6xYsWKJZ/ujr0Hl3yaC3Xh2acfGR5WfeNk0muc9Ppg8mscRX1r167dXlVTg8Yt+EBukhXAF4Ffq6pXk9wF3AFUd38n8EEgA55eDD5+8CPfOFW1CdgEMDU1VcO8JOmwLnl60ygP5F43fWR40i9ZC5Nf46TXB5Nf47jVt6DQT/IGeoF/b1V9CaCq9vWN/yzwQPdwD3Bu39PPAV7qhmdrlyQtg4WcvRPgbuDZqvr9vvaz+rq9D3imG94CXJPk1CTnAWuArwOPA2uSnJfkFHoHe7csTRmSpIVYyJb+u4DrgR1JnurafgO4Nsk76O2i2Q38EkBV7UxyH70DtK8DN1fVDwCS3AI8DJwEbK6qnUtYiyRpHgs5e+drDN5P/9Acz/kY8LEB7Q/N9TxJ0nB5GQZJaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQ+YN/STnJvnjJM8m2ZnkV7v2tyTZmuT57n5l154kn0yyK8nTSd7ZN60bu/7PJ7lxeGVJkgY5eQF9Xgduraonk7wZ2J5kK3AT8EhVbUyyAdgAfAS4AljT3S4B7gIuSfIW4KPAFFDddLZU1StLXZTasWPvQW7a8OBI5r1743tHMl/peMy7pV9VL1fVk93wd4BngbOBq4B7um73AFd3w1cBn6uex4AzkpwFXA5sraoDXdBvBdYtaTWSpDmlqhbeOVkNPApcALxYVWf0jXulqlYmeQDYWFVf69ofofcXwDTwxqr63a79t4DXqurjR81jPbAeYNWqVRfNzMwsurj5HDp0iBUrViz5dHfsPbjk01yoC88+/cjwsOobJ/sPHGTfa6OZd/9rPSwtrMNJr3EU9a1du3Z7VU0NGreQ3TsAJFkBfBH4tap6NcmsXQe01RztP9xQtQnYBDA1NVXT09MLXcRjtm3bNoYx/VHtbgDYfd30keFh1TdOPnXv/dy5Y8Fv4yXV/1oPSwvrcNJrHLf6FnT2TpI30Av8e6vqS13zvm63Dd39/q59D3Bu39PPAV6ao12StEwWcvZOgLuBZ6vq9/tGbQEOn4FzI3B/X/sN3Vk8lwIHq+pl4GHgsiQruzN9LuvaJEnLZCF/F78LuB7YkeSpru03gI3AfUk+BLwIvL8b9xBwJbAL+B7wAYCqOpDkDuDxrt/vVNWBJalCkrQg84Z+d0B2th347xnQv4CbZ5nWZmDzsSygJGnp+ItcSWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDVkNP99QtKirR7lP+nx/wKf8NzSl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUkHlDP8nmJPuTPNPXdnuSvUme6m5X9o27LcmuJM8lubyvfV3XtivJhqUvRZI0n4Vs6f8hsG5A+yeq6h3d7SGAJOcD1wBv757zB0lOSnIS8BngCuB84NquryRpGc37j9Gr6tEkqxc4vauAmar6PvCtJLuAi7txu6rqBYAkM13fbxzzEkuSFi1VNX+nXug/UFUXdI9vB24CXgWeAG6tqleSfBp4rKo+3/W7G/hKN5l1VfXhrv164JKqumXAvNYD6wFWrVp10czMzHGUN7dDhw6xYsWKJZ/ujr0Hl3yaC3Xh2acfGR5WfeNk/4GD7HttNPPuf62HZdA6HJf311KZ9PfpKOpbu3bt9qqaGjRu3i39WdwF3AFUd38n8EEgA/oWg3cjDfy2qapNwCaAqampmp6eXuQizm/btm0MY/o3bXhwyae5ULuvmz4yPKz6xsmn7r2fO3cs9m18fPpf62EZtA7H5f21VCb9fTpu9S3q01JV+w4PJ/ks8ED3cA9wbl/Xc4CXuuHZ2iVJy2RRp2wmOavv4fuAw2f2bAGuSXJqkvOANcDXgceBNUnOS3IKvYO9Wxa/2JKkxZh3Sz/JF4Bp4Mwke4CPAtNJ3kFvF81u4JcAqmpnkvvoHaB9Hbi5qn7QTecW4GHgJGBzVe1c8mokSXNayNk71w5ovnuO/h8DPjag/SHgoWNaOknSkvIXuZLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1Jasi8/xhdkg5bveHBJZ/mrRe+zk3zTHf3xvcu+Xxb5Za+JDVkorf059sqWcgWhiRNErf0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIfOGfpLNSfYneaav7S1JtiZ5vrtf2bUnySeT7ErydJJ39j3nxq7/80luHE45kqS5LGRL/w+BdUe1bQAeqao1wCPdY4ArgDXdbT1wF/S+JICPApcAFwMfPfxFIUlaPvOGflU9Chw4qvkq4J5u+B7g6r72z1XPY8AZSc4CLge2VtWBqnoF2MqPfpFIkoYsVTV/p2Q18EBVXdA9/puqOqNv/CtVtTLJA8DGqvpa1/4I8BFgGnhjVf1u1/5bwGtV9fEB81pP768EVq1addHMzMyii9ux9+Cc41e9Cfa9tujJj6ULzz79yPChQ4dYsWLFCJdm+PYfODiyddj/Wg/LoHU43/v6RLOQz+FyvNbDMorP4dq1a7dX1dSgcUt9lc0MaKs52n+0sWoTsAlgamqqpqenF70w811B89YLX+fOHZN1odHd100fGd62bRvH8/qdCD517/0jW4f9r/WwDFqHk3Zl2IV8DpfjtR6WcfscLvbsnX3dbhu6+/1d+x7g3L5+5wAvzdEuSVpGiw39LcDhM3BuBO7va7+hO4vnUuBgVb0MPAxclmRldwD3sq5NkrSM5v27OMkX6O2TPzPJHnpn4WwE7kvyIeBF4P1d94eAK4FdwPeADwBU1YEkdwCPd/1+p6qOPjgsSRqyeUO/qq6dZdR7BvQt4OZZprMZ2HxMSydJWlL+IleSGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXk5FEvgCTNZ/WGB0c2790b3zuyeQ+DW/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0JekhhxX6CfZnWRHkqeSPNG1vSXJ1iTPd/cru/Yk+WSSXUmeTvLOpShAkrRwS7Glv7aq3lFVU93jDcAjVbUGeKR7DHAFsKa7rQfuWoJ5S5KOwTB271wF3NMN3wNc3df+uep5DDgjyVlDmL8kaRapqsU/OfkW8ApQwH+uqk1J/qaqzujr80pVrUzyALCxqr7WtT8CfKSqnjhqmuvp/SXAqlWrLpqZmVn08u3Ye3DO8aveBPteW/Tkx9KFZ59+ZPjQoUOsWLFihEszfPsPHBzZOux/rYdl0Dqc7319ohn3z+HxrudRfA7Xrl27vW/vyw853guuvauqXkryE8DWJH85R98MaPuRb5yq2gRsApiamqrp6elFL9xN81yk6dYLX+fOHZN1zbnd100fGd62bRvH8/qdCD517/0jW4f9r/WwDFqH872vTzTj/jk83vU8bp/D49q9U1Uvdff7gS8DFwP7Du+26e73d933AOf2Pf0c4KXjmb8k6dgsOvSTnJbkzYeHgcuAZ4AtwI1dtxuB+7vhLcAN3Vk8lwIHq+rlRS+5JOmYHc/fVKuALyc5PJ3/WlV/lORx4L4kHwJeBN7f9X8IuBLYBXwP+MBxzFuStAiLDv2qegH46QHtfw28Z0B7ATcvdn6SpOPnL3IlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktSQRf9jdKl1qzc8OPR53Hrh69y0DPNRO9zSl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGrLsoZ9kXZLnkuxKsmG55y9JLVvW0E9yEvAZ4ArgfODaJOcv5zJIUsuWe0v/YmBXVb1QVX8LzABXLfMySFKzUlXLN7PkF4B1VfXh7vH1wCVVdUtfn/XA+u7hPwWeG+IinQl8e4jTH7VJrw8mv8ZJrw8mv8ZR1PdPqurHB41Y7uvpZ0DbD33rVNUmYNOyLEzyRFVNLce8RmHS64PJr3HS64PJr3Hc6lvu3Tt7gHP7Hp8DvLTMyyBJzVru0H8cWJPkvCSnANcAW5Z5GSSpWcu6e6eqXk9yC/AwcBKwuap2LucyHGVZdiON0KTXB5Nf46TXB5Nf41jVt6wHciVJo+UvciWpIYa+JDVk4kI/yeYk+5M809d2e5K9SZ7qbld27W9Ick+SHUmeTXJb33PG8nIRg+rr2n+lW96dSX6vr/22robnklze137C15fk55Ns79bf9iTv7ut/Ude+K8knkww6XXgkjnUdduP+cZJDSX69r+2EX4dd+08l+bOufUeSN3btY7kOj/E9On4ZU1UTdQP+BfBO4Jm+ttuBXx/Q9xeBmW74x4DdwGp6B5m/CfwkcArwF8D5o65tjvrWAl8FTu0e/0R3f3637KcC53U1nTRB9f0M8NZu+AJgb99zvg78M3q/DfkKcMWoa1tMjX3jvwj8t8Pv4wlahycDTwM/3T3+R8BJ47wOj7G+scuYidvSr6pHgQML7Q6cluRk4E3A3wKvMsaXi5ilvl8GNlbV97s++7v2q+i94b5fVd8CdtGrbSLqq6o/r6rDv/PYCbwxyalJzgL+YVX9WfU+bZ8Drl6eCuZ3jOuQJFcDL9Cr8bCJWIfAZcDTVfUXXftfV9UPxnkdHmN9Y5cxExf6c7glydPdn2Yru7b/DnwXeBl4Efh4VR0Azgb+T99z93Rt4+ptwM8l+V9J/iTJz3bts9UxKfX1+1fAn3cfurPp1XTYuNcHs9SY5DTgI8BvH9V/Utbh24BK8nCSJ5P8u679RFuHs9U3dhmz3JdhGJW7gDvofeveAdwJfJDet+0PgLcCK4H/meSrLOByEWPmZHrLfynws8B9SX6S2esY9GV/wtXXbQGS5O3Af6S31Qgn3vqD2dfhbwOfqKpDR+3SPtFqnK2+k4F/3rV9D3gkyXZ6W8NHOxHrG7uMaSL0q2rf4eEknwUe6B7+IvBHVfV3wP4kfwpM0fsGPpEuF7EH+FIXgl9P8vf0LvI012UvJqG+/5vkHODLwA1V9c2+/uf0PX/c64PZa7wE+IXuwOAZwN8n+X/AdiZjHe4B/qSqvg2Q5CF6+8s/z4m1Dmerb+wypondO93+wcPeBxw+6v4i8O70nEbvW/ovOfEuF/E/gHcDJHkbvQND36a3zNd0+7nPA9bQOzg2EfUlOQN4ELitqv70cOeqehn4TpJLuzM+bgDuX/7FPiYDa6yqn6uq1VW1GvhPwH+oqk8zIeuQ3q/zfyrJj3X7vf8l8I0TcB3OVt/4Zcyoj4Qv9Q34Ar39Z39H79v3Q8B/AXbQO0tgC3BW13cFvTMidgLfAP5t33SuBP6K3hH23xx1XfPUdwq9LaNngCeBd/f1/82uhufoO/thEuoD/j29/aVP9d0OnzUx1fX/JvBpul+fj8PtWNdh3/Nup+8stElYh13/f919Bp8Bfq+vfSzX4TG+R8cuY7wMgyQ1pIndO5KkHkNfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNeT/AyPfHiqz0BPhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_contracts.startY.hist(bins=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking at empirical distributions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1ff56cf8>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAUC0lEQVR4nO3df6zldX3n8edrB7Wu1IIFb6YzuIPJaArSzpYbJHG3udRWBzAFN3UXwgqjbkYbSDQ7m92h2wRXQ8Lu1to1dWlGmYCpy0ikykRw7ZT1BjeRCqOUAZFlwKlcZjITxYJXDZuh7/3jfO/e0+H+POfOmeF+no/k5Jzv5/vrc94Mr/O9n+/3fE+qCklSG/7Rie6AJGl0DH1JaoihL0kNMfQlqSGGviQ15JQT3YHFnHHGGbVhw4Zlr/fTn/6U17zmNSvfoZcZ69BjHWZZi57VXIe9e/f+sKrOnGveSR/6GzZs4MEHH1z2epOTk0xMTKx8h15mrEOPdZhlLXpWcx2S/O188xzekaSGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhpz038g9UTZsv3vB+QduunREPZGkleORviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWrIoqGfZGeSI0ke6Wv7QpKHuseBJA917RuS/Lxv3p/1rXN+kn1J9if5VJIcn7ckSZrPUq7TvxX4U+BzMw1V9a9mXif5BPBc3/JPVtWmObZzM7AVuB+4B9gMfHX5XZYkDWrRI/2qug94dq553dH6vwRuX2gbSdYCr62qb1ZV0fsAuXz53ZUkDWPYb+T+c+BwVT3R13Z2ku8AzwN/WFXfANYBU33LTHVtc0qyld5fBYyNjTE5Obnsjk1PTw+03oxt5x1dcP4w2x6lYeuwWliHWdaip9U6DBv6V/IPj/IPAW+oqh8lOR/4cpJzgbnG72u+jVbVDmAHwPj4eA3y48XD/ujxlsVuw3DV4NsepdX848/LYR1mWYueVuswcOgnOQX4F8D5M21V9QLwQvd6b5IngTfRO7Jf37f6euDgoPuWJA1mmEs2fxv4XlX9/2GbJGcmWdO9fiOwEXiqqg4BP0lyYXce4GrgriH2LUkawFIu2bwd+Cbw5iRTST7QzbqCl57A/U3g4SR/A3wR+FBVzZwE/n3gs8B+4Em8ckeSRm7R4Z2qunKe9i1ztN0J3DnP8g8Cb1lm/yRJK8hv5EpSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNWcoPo+9MciTJI31tH03yTJKHusclffOuT7I/yeNJ3tnXvrlr259k+8q/FUnSYpZypH8rsHmO9k9W1abucQ9AknOAK4Bzu3X+e5I1SdYAnwYuBs4BruyWlSSN0CmLLVBV9yXZsMTtXQbsqqoXgO8n2Q9c0M3bX1VPASTZ1S373WX3WJI0sEVDfwHXJbkaeBDYVlU/BtYB9/ctM9W1ATx9TPtb59twkq3AVoCxsTEmJyeX3bnp6emB1pux7byjC84fZtujNGwdVgvrMMta9LRah0FD/2bg40B1z58A3g9kjmWLuYeRar6NV9UOYAfA+Ph4TUxMLLuDk5OTDLLejC3b715w/oGrBt/2KA1bh9XCOsyyFj2t1mGg0K+qwzOvk3wG+Eo3OQWc1bfoeuBg93q+dknSiAx0yWaStX2T7wZmruzZDVyR5FVJzgY2At8CHgA2Jjk7ySvpnezdPXi3JUmDWPRIP8ntwARwRpIp4AZgIskmekM0B4APAlTVo0nuoHeC9ihwbVW92G3nOuBrwBpgZ1U9uuLvRpK0oKVcvXPlHM23LLD8jcCNc7TfA9yzrN5JklaU38iVpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktSQRX8uUXPbsP3ueecduOnSEfZEkpZu0SP9JDuTHEnySF/bf03yvSQPJ/lSktO69g1Jfp7koe7xZ33rnJ9kX5L9ST6VJMfnLUmS5rOU4Z1bgc3HtO0B3lJVvwb8H+D6vnlPVtWm7vGhvvabga3Axu5x7DYlScfZoqFfVfcBzx7T9pdVdbSbvB9Yv9A2kqwFXltV36yqAj4HXD5YlyVJg1qJMf33A1/omz47yXeA54E/rKpvAOuAqb5lprq2OSXZSu+vAsbGxpicnFx2p6anpwdab8a2844uvtA8htnvShu2DquFdZhlLXparcNQoZ/kPwJHgc93TYeAN1TVj5KcD3w5ybnAXOP3Nd92q2oHsANgfHy8JiYmlt23yclJBllvxpYFTtQu5sBVg+93pQ1bh9XCOsyyFj2t1mHg0E9yDfAu4O3dkA1V9QLwQvd6b5IngTfRO7LvHwJaDxwcdN+SpMEMdJ1+ks3AfwB+t6p+1td+ZpI13es30jth+1RVHQJ+kuTC7qqdq4G7hu69JGlZFj3ST3I7MAGckWQKuIHe1TqvAvZ0V17e312p85vAx5IcBV4EPlRVMyeBf5/elUCvBr7aPSRJI7Ro6FfVlXM03zLPsncCd84z70HgLcvqnSRpRXkbBklqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNafY3chf6jVtJWq080pekhhj6ktSQZod3jqfFho4O3HTpiHoiSf+QR/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ1ZUugn2ZnkSJJH+tpel2RPkie659O79iT5VJL9SR5O8ht961zTLf9EkmtW/u1Ikhay1CP9W4HNx7RtB+6tqo3Avd00wMXAxu6xFbgZeh8SwA3AW4ELgBtmPigkSaOxpNCvqvuAZ49pvgy4rXt9G3B5X/vnqud+4LQka4F3Anuq6tmq+jGwh5d+kEiSjqNhvpw1VlWHAKrqUJLXd+3rgKf7lpvq2uZrf4kkW+n9lcDY2BiTk5PL7tz09PSC62077+iyt7lSBnk/g1qsDq2wDrOsRU+rdTge38jNHG21QPtLG6t2ADsAxsfHa2JiYtmdmJycZKH1tpzIG67t++m8s1b627qL1aEV1mGWtehptQ7DXL1zuBu2oXs+0rVPAWf1LbceOLhAuyRpRIYJ/d3AzBU41wB39bVf3V3FcyHwXDcM9DXgHUlO707gvqNrkySNyJKGd5LcDkwAZySZoncVzk3AHUk+APwAeE+3+D3AJcB+4GfA+wCq6tkkHwce6Jb7WFUde3JYknQcLSn0q+rKeWa9fY5lC7h2nu3sBHYuuXeSpBXlN3IlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhgwc+knenOShvsfzST6S5KNJnulrv6RvneuT7E/yeJJ3rsxbkCQt1ZJ+GH0uVfU4sAkgyRrgGeBLwPuAT1bVH/Uvn+Qc4ArgXOBXgL9K8qaqenHQPkiSlmelhnfeDjxZVX+7wDKXAbuq6oWq+j6wH7hghfYvSVqClQr9K4Db+6avS/Jwkp1JTu/a1gFP9y0z1bVJkkYkVTXcBpJXAgeBc6vqcJIx4IdAAR8H1lbV+5N8GvhmVf15t94twD1Vdecc29wKbAUYGxs7f9euXcvu1/T0NKeeeuq88/c989yytzkK5637pRXd3mJ1aIV1mGUtelZzHS666KK9VTU+17yBx/T7XAx8u6oOA8w8AyT5DPCVbnIKOKtvvfX0Pixeoqp2ADsAxsfHa2JiYtmdmpycZKH1tmy/e9nbHIUDV02s6PYWq0MrrMMsa9HTah1WYnjnSvqGdpKs7Zv3buCR7vVu4Iokr0pyNrAR+NYK7F+StERDHekn+cfA7wAf7Gv+L0k20RveOTAzr6oeTXIH8F3gKHCtV+5I0mgNFfpV9TPgl49pe+8Cy98I3DjMPiVJg/MbuZLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDhg79JAeS7EvyUJIHu7bXJdmT5Inu+fSuPUk+lWR/koeT/Maw+5ckLd1KHelfVFWbqmq8m94O3FtVG4F7u2mAi4GN3WMrcPMK7V+StATHa3jnMuC27vVtwOV97Z+rnvuB05KsPU59kCQdYyVCv4C/TLI3ydaubayqDgF0z6/v2tcBT/etO9W1SZJG4JQV2MbbqupgktcDe5J8b4FlM0dbvWSh3ofHVoCxsTEmJyeX3anp6ekF19t23tFlb3MUBnmvC1msDq2wDrOsRU+rdRg69KvqYPd8JMmXgAuAw0nWVtWhbvjmSLf4FHBW3+rrgYNzbHMHsANgfHy8JiYmlt2vyclJFlpvy/a7l73NUThw1cSKbm+xOrTCOsyyFj2t1mGo4Z0kr0nyizOvgXcAjwC7gWu6xa4B7upe7wau7q7iuRB4bmYYSJJ0/A17pD8GfCnJzLb+R1X9zyQPAHck+QDwA+A93fL3AJcA+4GfAe8bcv+SpGUYKvSr6ing1+do/xHw9jnaC7h2mH1KkgbnN3IlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNWQlbq2sFbRhkbt/Hrjp0hH1RNJq5JG+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1ZODQT3JWkq8neSzJo0k+3LV/NMkzSR7qHpf0rXN9kv1JHk/yzpV4A5KkpRvmNgxHgW1V9e0kvwjsTbKnm/fJqvqj/oWTnANcAZwL/ArwV0neVFUvDtEHSdIyDHykX1WHqurb3eufAI8B6xZY5TJgV1W9UFXfB/YDFwy6f0nS8qWqht9IsgG4D3gL8G+BLcDzwIP0/hr4cZI/Be6vqj/v1rkF+GpVfXGO7W0FtgKMjY2dv2vXrmX3aXp6mlNPPXXe+fueeW7Z2zwZnLfulxacf+z7Gns1HP750tZdzRb799ASa9Gzmutw0UUX7a2q8bnmDX2XzSSnAncCH6mq55PcDHwcqO75E8D7gcyx+pyfOFW1A9gBMD4+XhMTE8vu1+TkJAutt2WRu1merA5cNbHg/GPf17bzjvKJfacsad3VbLF/Dy2xFj2t1mGoq3eSvIJe4H++qv4CoKoOV9WLVfX3wGeYHcKZAs7qW309cHCY/UuSlmeYq3cC3AI8VlV/3Ne+tm+xdwOPdK93A1ckeVWSs4GNwLcG3b8kafmGGd55G/BeYF+Sh7q2PwCuTLKJ3tDNAeCDAFX1aJI7gO/Su/LnWq/ckaTRGjj0q+p/M/c4/T0LrHMjcOOg+5QkDcdv5EpSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGDH0bBq0eGxa4NcWBmy4dYU8kHS+rOvQXCrGXq9X4niSNjsM7ktQQQ1+SGmLoS1JDVvWYvkZnsXMNngiWTg4e6UtSQzzS16o28xfItvOOzvlraf4FotZ4pC9JDTH0Jakhhr4kNcQxfakx+555bs7zG+A5jhYY+hqJFu/r0+J7PlnN9d9i5uR+a/8tRh76STYD/w1YA3y2qm4adR9aNcx9e07kPX9O1u8AeB8kvRyNNPSTrAE+DfwOMAU8kGR3VX13lP3QyeVkDc+TtV/SMEZ9pH8BsL+qngJIsgu4DDD0NbCX418wx3O/w/zlM0y/FtvvybrtE+lEDAGmqo7LhufcWfJ7wOaq+jfd9HuBt1bVdccstxXY2k2+GXh8gN2dAfxwiO6uFtahxzrMshY9q7kO/6SqzpxrxqiP9DNH20s+dapqB7BjqB0lD1bV+DDbWA2sQ491mGUtelqtw6iv058CzuqbXg8cHHEfJKlZow79B4CNSc5O8krgCmD3iPsgSc0a6fBOVR1Nch3wNXqXbO6sqkeP0+6GGh5aRaxDj3WYZS16mqzDSE/kSpJOLO+9I0kNMfQlqSGrLvSTbE7yeJL9Sbaf6P6MUpKdSY4keaSv7XVJ9iR5ons+/UT2cRSSnJXk60keS/Jokg937U3VIskvJPlWkr/p6vCfuvazk/x1V4cvdBdVNCHJmiTfSfKVbrq5Wqyq0O+7zcPFwDnAlUnOObG9Gqlbgc3HtG0H7q2qjcC93fRqdxTYVlW/ClwIXNv9O2itFi8Av1VVvw5sAjYnuRD4z8Anuzr8GPjACezjqH0YeKxvurlarKrQp+82D1X1f4GZ2zw0oaruA549pvky4Lbu9W3A5SPt1AlQVYeq6tvd65/Q+598HY3Vonqmu8lXdI8Cfgv4Yte+6uswI8l64FLgs910aLAWqy301wFP901PdW0tG6uqQ9ALQ+D1J7g/I5VkA/BPgb+mwVp0wxkPAUeAPcCTwN9V1dFukZb+H/kT4N8Df99N/zIN1mK1hf6SbvOgNiQ5FbgT+EhVPX+i+3MiVNWLVbWJ3rffLwB+da7FRtur0UvyLuBIVe3tb55j0VVfi9X2Iyre5uGlDidZW1WHkqyld8S36iV5Bb3A/3xV/UXX3GQtAKrq75JM0jvHcVqSU7oj3Fb+H3kb8LtJLgF+AXgtvSP/5mqx2o70vc3DS+0GruleXwPcdQL7MhLdWO0twGNV9cd9s5qqRZIzk5zWvX418Nv0zm98Hfi9brFVXweAqrq+qtZX1QZ6ufC/quoqGqzFqvtGbvdJ/ifM3ubhxhPcpZFJcjswQe+WsYeBG4AvA3cAbwB+ALynqo492buqJPlnwDeAfcyO3/4BvXH9ZmqR5NfonZxcQ+8A746q+liSN9K7yOF1wHeAf11VL5y4no5Wkgng31XVu1qsxaoLfUnS/Fbb8I4kaQGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWrI/wP7x47xFCqAMQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_contracts[df_contracts.annual_salary < 50].annual_salary.hist(bins=40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a20297198>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQq0lEQVR4nO3df6zddX3H8edr4KahhpagDQG2y5Zm0dGNwQ2QuCy3c8MCf4CJLBKmxWnqH5Boxh90Jgbmj6RZ1G1ujq2OxpKpd2TqaICNNY03jD9QWocUZI5OO1batHGF6lXjUn3vj/Ptdnu9v3t/nfN5PpKbc877fM73fN759r7O936/3/NtqgpJUlt+ZqUnIElafoa/JDXI8JekBhn+ktQgw1+SGnTuSk9gJhdeeGENDQ2dUfv+97/PeeedtzITWkL21X8GtbdB7QsGt7fJfe3fv/87VfW6mV6zqsN/aGiIffv2nVEbGxtjZGRkZSa0hOyr/wxqb4PaFwxub5P7SvKfs73G3T6S1CDDX5IaZPhLUoMMf0lqkOEvSQ0y/CWpQYa/JDXI8JekBhn+ktSgVf0NX2kpDG17ZF7jD22/cYlmIq0ct/wlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMMf0lq0Kzhn+TSJF9O8nyS55K8r6tfkGRPkhe623VdPUk+meRgkmeSXDlhWVu68S8k2bJ0bUmSZjKXLf9TwF1V9QbgWuCOJG8EtgF7q2oDsLd7DHA9sKH72QrcB70PC+Ae4BrgauCe0x8YkqTlNWv4V9XRqvpad/97wPPAxcBNwK5u2C7g5u7+TcAD1fMksDbJRcBbgD1VdaKqXgb2AJsXtRtJ0pykquY+OBkCHgcuB16sqrUTnnu5qtYleRjYXlVPdPW9wN3ACPDqqvpIV/8g8MOq+tik99hK7y8G1q9ff9Xo6OgZcxgfH2fNmjXz67IP2NfyOfDSyXmN33jx+VPWV2Nvi2FQ+4LB7W1yX5s2bdpfVcMzvWbO/4dvkjXAF4D3V9V3k0w7dIpazVA/s1C1A9gBMDw8XCMjI2c8PzY2xuTaILCv5XP7fP8P39tGpqyvxt4Ww6D2BYPb20L6mtPZPkleRS/4P1tVX+zKx7rdOXS3x7v6YeDSCS+/BDgyQ12StMzmcrZPgPuB56vqExOe2g2cPmNnC/DQhPo7u7N+rgVOVtVR4DHguiTrugO913U1SdIym8tunzcB7wAOJHm6q30A2A48mOTdwIvALd1zjwI3AAeBHwDvAqiqE0k+DDzVjftQVZ1YlC4kSfMya/h3B26n28H/5inGF3DHNMvaCeyczwQlSYvPb/hKUoMMf0lqkOEvSQ0y/CWpQYa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMMf0lqkOEvSQ0y/CWpQYa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1yPCXpAbNGv5JdiY5nuTZCbV7k7yU5Onu54YJz/1hkoNJvpnkLRPqm7vawSTbFr8VSdJczWXL/zPA5inqf1JVV3Q/jwIkeSPwduBXutf8ZZJzkpwDfAq4HngjcGs3VpK0As6dbUBVPZ5kaI7LuwkYraofAd9OchC4unvuYFV9CyDJaDf2G/OesSTprKWqZh/UC/+Hq+ry7vG9wO3Ad4F9wF1V9XKSvwCerKq/7cbdD/xjt5jNVfWerv4O4JqqunOK99oKbAVYv379VaOjo2c8Pz4+zpo1a+bb56pnX8vnwEsn5zV+48XnT1lfjb0thkHtCwa3t8l9bdq0aX9VDc/0mlm3/KdxH/BhoLrbjwO/D2SKscXUu5em/NSpqh3ADoDh4eEaGRk54/mxsTEm1waBfS2f27c9Mq/xh24bmbK+GntbDIPaFwxubwvpa0HhX1XHTt9P8mng4e7hYeDSCUMvAY5096erS5KW2YJO9Uxy0YSHbwVOnwm0G3h7kp9LchmwAfgq8BSwIcllSX6W3kHh3QuftiTpbMy65Z/k88AIcGGSw8A9wEiSK+jtujkEvBegqp5L8iC9A7mngDuq6sfdcu4EHgPOAXZW1XOL3o0kaU7mcrbPrVOU759h/EeBj05RfxR4dF6zkyQtCb/hK0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMMf0lqkOEvSQ1a0H/gLi2VoW2PzGv8oe03LtFMpMHmlr8kNcjwl6QGudtHfW2+u4kk9bjlL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMMf0lqkOEvSQ2aNfyT7ExyPMmzE2oXJNmT5IXudl1XT5JPJjmY5JkkV054zZZu/AtJtixNO5KkuZjLlv9ngM2TatuAvVW1AdjbPQa4HtjQ/WwF7oPehwVwD3ANcDVwz+kPDEnS8ps1/KvqceDEpPJNwK7u/i7g5gn1B6rnSWBtkouAtwB7qupEVb0M7OGnP1AkScskVTX7oGQIeLiqLu8ev1JVayc8/3JVrUvyMLC9qp7o6nuBu4ER4NVV9ZGu/kHgh1X1sSneayu9vxpYv379VaOjo2c8Pz4+zpo1a+bf6SpnXz0HXjq5hLNZmI0Xnz9l3XXWfwa1t8l9bdq0aX9VDc/0mnMXeQ6ZolYz1H+6WLUD2AEwPDxcIyMjZzw/NjbG5NogsK+e27c9snSTWaBDt41MWXed9Z9B7W0hfS00/I8luaiqjna7dY539cPApRPGXQIc6eojk+pjC3xvaVkNTfOBdNfGU1N+WB3afuNST0k6aws91XM3cPqMnS3AQxPq7+zO+rkWOFlVR4HHgOuSrOsO9F7X1SRJK2DWLf8kn6e31X5hksP0ztrZDjyY5N3Ai8At3fBHgRuAg8APgHcBVNWJJB8GnurGfaiqJh9EliQtk1nDv6puneapN08xtoA7plnOTmDnvGYnSVoSi33AV2redMcIpuMxAq0EL+8gSQ0y/CWpQYa/JDXIff5aUgdeOrkqv7gltc4tf0lqkOEvSQ0y/CWpQYa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IadO5KT0DS/Axte2Re4w9tv3GJZqJ+5pa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ16KzCP8mhJAeSPJ1kX1e7IMmeJC90t+u6epJ8MsnBJM8kuXIxGpAkzd9ibPlvqqorqmq4e7wN2FtVG4C93WOA64EN3c9W4L5FeG9J0gIsxW6fm4Bd3f1dwM0T6g9Uz5PA2iQXLcH7S5Jmkapa+IuTbwMvAwX8dVXtSPJKVa2dMOblqlqX5GFge1U90dX3AndX1b5Jy9xK7y8D1q9ff9Xo6OgZ7zk+Ps6aNWsWPOfValD7On7iJMd+uNKzWBrrX8Oi9Lbx4vPnNf7ASyeXdPmD+m8RBre3yX1t2rRp/4S9MVM626t6vqmqjiR5PbAnyb/NMDZT1H7qk6eqdgA7AIaHh2tkZOSM58fGxphcGwSD2teff/YhPn5gMC8ee9fGU4vS26HbRuY1/vb5XtVznssf1H+LMLi9LaSvs9rtU1VHutvjwJeAq4Fjp3fndLfHu+GHgUsnvPwS4MjZvL8kaWEWHP5Jzkvy2tP3geuAZ4HdwJZu2Bbgoe7+buCd3Vk/1wInq+rogmcuSVqws/mbdT3wpSSnl/O5qvqnJE8BDyZ5N/AicEs3/lHgBuAg8APgXWfx3pKks7Dg8K+qbwG/NkX9v4E3T1Ev4I6Fvp8kafH4DV9JapDhL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUoMH83r3UR4bmebmGpV7+XRtPMbI0U9Eq4pa/JDXI8JekBhn+ktQgw1+SGmT4S1KDDH9JapDhL0kNMvwlqUGGvyQ1yPCXpAYZ/pLUIMNfkhpk+EtSgwx/SWqQ4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5IaZPhLUoMMf0lq0LkrPQH1l6Ftj8xr/F0bl2giks6KW/6S1CDDX5IaZPhLUoMMf0lqkOEvSQ3ybJ/GzffsHUmDwfCXdNbmuxFxaPuNSzQTzZXhL2nV88Nl8S17+CfZDPwZcA7wN1W1fbnnIGlm7g4cfMsa/knOAT4F/A5wGHgqye6q+sZyzqNf+Asoaaks95b/1cDBqvoWQJJR4CbA8JcastQbNtMt/66Np7h9kd57vruWVtuuq1TVkr7BGW+WvA3YXFXv6R6/A7imqu6cMGYrsLV7+MvANyct5kLgO8sw3eVmX/1nUHsb1L5gcHub3NcvVNXrZnrBcm/5Z4raGZ8+VbUD2DHtApJ9VTW82BNbafbVfwa1t0HtCwa3t4X0tdxf8joMXDrh8SXAkWWegyQ1b7nD/ylgQ5LLkvws8HZg9zLPQZKat6y7farqVJI7gcfoneq5s6qem+dipt0l1Ofsq/8Mam+D2hcMbm/z7mtZD/hKklYHL+wmSQ0y/CWpQX0T/kk2J/lmkoNJtq30fBZTkkNJDiR5Osm+lZ7PQiXZmeR4kmcn1C5IsifJC93tupWc40JN09u9SV7q1tvTSW5YyTkuRJJLk3w5yfNJnkvyvq7e1+tthr76ep0leXWSryb5etfXH3X1y5J8pVtff9edUDPzsvphn393WYh/Z8JlIYBbB+WyEEkOAcNV1ddfPknym8A48EBVXd7V/hg4UVXbuw/tdVV190rOcyGm6e1eYLyqPraSczsbSS4CLqqqryV5LbAfuBm4nT5ebzP09bv08TpLEuC8qhpP8irgCeB9wB8AX6yq0SR/BXy9qu6baVn9suX/f5eFqKr/AU5fFkKrSFU9DpyYVL4J2NXd30XvF7DvTNNb36uqo1X1te7+94DngYvp8/U2Q199rXrGu4ev6n4K+C3g77v6nNZXv4T/xcB/TXh8mAFYkRMU8M9J9neXtxgk66vqKPR+IYHXr/B8FtudSZ7pdgv11a6RyZIMAb8OfIUBWm+T+oI+X2dJzknyNHAc2AP8B/BKVZ3qhswpH/sl/Ge9LESfe1NVXQlcD9zR7WLQ6ncf8EvAFcBR4OMrO52FS7IG+ALw/qr67krPZ7FM0Vffr7Oq+nFVXUHvCglXA2+Yathsy+mX8B/oy0JU1ZHu9jjwJXordFAc6/a/nt4Pe3yF57NoqupY94v4E+DT9Ol66/YdfwH4bFV9sSv3/Xqbqq9BWWcAVfUKMAZcC6xNcvpLu3PKx34J/4G9LESS87oDUiQ5D7gOeHbmV/WV3cCW7v4W4KEVnMuiOh2OnbfSh+utO4B4P/B8VX1iwlN9vd6m66vf11mS1yVZ291/DfDb9I5nfBl4WzdsTuurL872AehOyfpT/v+yEB9d4SktiiS/SG9rH3qX2/hcv/aW5PPACL3Lyx4D7gH+AXgQ+HngReCWquq7A6fT9DZCb/dBAYeA957eT94vkvwG8C/AAeAnXfkD9PaP9+16m6GvW+njdZbkV+kd0D2H3sb7g1X1oS5HRoELgH8Ffq+qfjTjsvol/CVJi6dfdvtIkhaR4S9JDTL8JalBhr8kNcjwl6QGGf6S1CDDX5Ia9L+FZ0pTw6yERgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_contracts[df_contracts.a_age < 30].a_age.hist(bins=25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Two very important distributions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Normal\n",
    "\n",
    "Also known as Gaussian, is a bell-shaped distribution with mass around the mean and exponentially decaying on the sides. It is fully characterized by the mean (center of mass) and standard deviation (spread).\n",
    "\n",
    "https://en.wikipedia.org/wiki/Normal_distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1b0a5d68>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD4CAYAAADrRI2NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXjU933g8fdnRhrdErrQDUggTmODLYMdx44TOzZOUnDSHHa3idPN1utt3GST3bbOHsmu82SbuLtpu62bxknczYWJ4+RpiIODbfCFbTACA46QBJIAIdAtdJ8z89k/NJCxEGgEI/3m+LyeR49+13fmo3k0n/nO9/c9RFUxxhgTP1xOB2CMMWZ+WeI3xpg4Y4nfGGPijCV+Y4yJM5b4jTEmziQ4HcBUeXl5umTJEqfDMMaYqHLgwIEuVc0P5dqIS/xLliyhurra6TCMMSaqiMipUK+1ph5jjIkzlviNMSbOWOI3xpg4Y4nfGGPijCV+Y4yJM5b4jTEmzljiN8aYOGOJ3xhj4owlfmOMiTMRN3LXmPm2dV/zRcf+aOMiByIxZn5Yjd+YaagqbX2j1LX14/fbKnUmtoRU4xeRTcDfA27g+6r6zUtc93Hg58CNqlodOPYV4HOAD/iCqu4MR+DGzIUxr4/fHGnlb188RufAGAC5aR5urczjwduWcuh070Vl7NuBiTYzJn4RcQOPAx8EWoD9IrJdVY9OuS4D+AKwL+jYauA+YA1QDLwoIstV1Re+P8GY8Dg3NM6P956ivX+UzeuKWV+2gPTkRF5v6GJ3XQfPHmnltuX53L4inwSXfVk20SuU/94NQIOqNqnqOLAN2DLNdV8HHgNGg45tAbap6piqngAaAo9nTETp6B/l8Zcb6B0Z57PvWcLG8lw8CW7GvX5uXJLDn79/GdeUZLG7roPvvdrE6ITVXUz0CiXxlwCng/ZbAscuEJH1QJmqPjvbsoHyD4pItYhUd3Z2hhS4MeHi8yvPHGwB4M/et4zKgoyLrklNSuCTVWXcd2MZZ3pH+NGbJxn3+uc5UmPCI5TEL9Mcu3C3S0RcwN8C/2m2ZS8cUH1CVatUtSo/P6R1BIwJmz0NXbScG2HzdcXkZSRd9tprSxfwyaoyTnUP85N9p/D6Lfmb6BPKzd0WoCxovxQ4G7SfAVwDvCwiAIXAdhHZHEJZY+bV1K6b7f2jvFjbzpriTNaWZIX0GNeWLmDC5+cXB8/wcn0nn7l5yRxEaszcCaXGvx+oFJFyEfEwebN2+/mTqtqnqnmqukRVlwB7gc2BXj3bgftEJElEyoFK4K2w/xXGXKHfHGnF43ax+bpiAhWXkNywOId1ZQt4ub6D2tb+OYzQmPCbMfGrqhd4GNgJ1AJPq2qNiDwaqNVfrmwN8DRwFPgt8Hnr0WMixZlzIzR0DvK+5flkJCfOuvxH1haR4kngL545jNdnTT4meoTUj19VdwA7phz76iWuvX3K/jeAb1xhfMbMmVeOd5Kc6GJDec4VlU9NSmDLdcVsfauZJ18/wYO3LQ1zhMbMDeuMbOJS1+AYNWf62FieS3Ki+4of55qSLG5bns93X2liZNy+zJroYInfxKXXjnfhdgnvWZp71Y/18PuX0T00zs/2XzznjzGRyBK/iTtDY17ebj7H9Yuyr6htf6oN5TlULc7me6+dYMLa+k0UsNk5Tdw5dLoXr1+5KQy1fZjsIrqmOJPqU+f4yi/e4frF2YDN4WMil9X4Tdw52HyOkgUpFGYmh+0xlxdkUJSVzCvHOlG12TxNZLPEb+LK2d4RWvtGL9TKw0VEuGVpHp2DY5zqHg7rYxsTbpb4TVw50HwOt0u4rjS0UbqzsaYkE4/bxcHmc2F/bGPCyRK/iRvjXj+HT/eyuiiTVE/4b28lJbi5piSTd8702QRuJqJZ4jdxY3ddO8PjPq5fFN5mnmDrF2Uz5vVz1KZxMBHMEr+JG786dJb0pASWLUyfs+coz0tjQWoib1tzj4lglvhNXBgc87K7roNrSjJxu0KfjG22XCKsL1tAQ8cgbX2jMxcwxgGW+E1c2FXbzpjXz9qSBXP+XOsXZaPArw/bDOQmMtkALhMXnj3SSkFmEotzU+f8ufLSkyjKSuYne0+RlvTut5gN6jKRwGr8Jub1j07wSn0nH1pbhGsWc+5fjTXFmTT3DNM/OjEvz2fMbFjiNzHvxaPtjPv8fOTa4nl7zjXFWSjYIi0mIlniNzHv2SOtFGcls75s7tv3z1uYkUReuoeas5b4TeQJKfGLyCYRqReRBhF5ZJrzD4nIOyJySET2iMjqwPElIjISOH5IRP453H+AMZfTPzrBnuNd3LO2CNcc9uaZSkRYU5xFU+cgw+PeeXteY0IxY+IXETfwOHAPsBq4/3xiD7JVVdeq6jrgMeDbQecaVXVd4OehcAVuTChequtg3OfnnmsK5/251xRn4leoaxuY9+c25nJCqfFvABpUtUlVx4FtwJbgC1Q1+PtsGmDTE5qI8HxNO3npSXM6WvdSShakkJWSSM2Zvnl/bmMuJ5TEXwKcDtpvCRx7FxH5vIg0Mlnj/0LQqXIReVtEXhGRW6d7AhF5UESqRaS6s7NzFuEbc2mjEz5equ/grjUF89rMc56IsLook4bOQZu7x0SUUBL/dO+Yi2r0qvq4qi4F/gr4b4HDrcAiVV0PfBnYKiKZ05R9QlWrVLUqPz8/9OiNuYw9x7sYHvexac38N/Oct6ookwmf0tAx6FgMxkwVSuJvAcqC9kuByw1J3AbcC6CqY6raHdg+ADQCy68sVGNm57c1bWQkJ3BTRXhW2roS5XlpJCe6bNI2E1FCSfz7gUoRKRcRD3AfsD34AhGpDNr9MHA8cDw/cHMYEakAKoGmcARuzOV4fX5erG3nzlUFeBKc67XsdgkrCjKoa+vHbytzmQgx45QNquoVkYeBnYAbeFJVa0TkUaBaVbcDD4vIncAEcA54IFD8NuBREfECPuAhVe2Ziz/EmGBvneyhd3iCu9cUOB0Kq4oyOdzSZytzmYgR0lw9qroD2DHl2FeDtr94iXK/AH5xNQEacyV21XbgSXBxa6Xz94yWF2TgFrFRvCZi2MhdE3NUlV217dxckXvRJGlOSE50U5GfRm1rvy3EbiKC8+8KY8KssXOIk93DXFu6gK37mp0OB5hs7tl++CwNHYNUFmQ4HY6Jc1bjNzFnV207ACsLIyfBriqa7MX8/NF2hyMxxhK/iUG7ajsoykpmQarH6VAuyEpJpGRBCi9Y4jcRwBK/iSm9w+NUn+qJqNr+eauKMjl0upeOfluS0TjLEr+JKS/Xd+JXWFl40QBxx60ONPe8WNvhcCQm3tnNXRP1gm/gbtvfTFpSAiXZKQ5GNL2CzCTKclJ44WibLcFoHGU1fhMz/Kocbx9kRUH6vC2xOBsiwgdXFfJ6YzdDYzZHv3GOJX4TM86cG2FkwhfR3SU/uLqAca+fV4/ZLLTGOZb4Tcw41j6AAJX56U6Hckk3LslmQWoiv61pczoUE8cs8ZuYcax9gNLsFFIjYLTupSS4Xdy9upAXj7YzOuFzOhwTpyzxm5gwPOal5dxIRDfznPeR64oYGvfxcr019xhnWOI3MaGhcxBlckK0SHdzRS45aR6ePXK5ZS2MmTuW+E1MONY+SEqim9II7MY5VYLbxaZrCtlV28HIuDX3mPlnid9EPVXleMcAyxZGZjfO6XxkbREjEz5219lgLjP/LPGbqNfeP8bAqJfKhZHbm2eqjRW55KUn8Zt3rLnHzL+QEr+IbBKRehFpEJFHpjn/kIi8IyKHRGSPiKwOOveVQLl6Ebk7nMEbA5Pt+wDLoijxu13Ch9ZONvcMjE44HY6JMzMm/sCauY8D9wCrgfuDE3vAVlVdq6rrgMeAbwfKrmZyjd41wCbgn86vwWtMuDR2DJKX7omo2ThDsWVdMWNePztrbMZOM79CqfFvABpUtUlVx4FtwJbgC1Q1eE25NOD8MkNbgG2qOqaqJ4CGwOMZExbjXj8nuoZYGsGDti7l+kXZlOWk8K9vn3E6FBNnQkn8JcDpoP2WwLF3EZHPi0gjkzX+L8ymrDFX6tDpXsZ9/qhq5jlPRPjouhJeb+yi3aZqNvMolMQ/XTeJixYOVdXHVXUp8FfAf5tNWRF5UESqRaS6s9MGtZjQvd7QhQAVedGX+AG2rC9BFX592G7ymvkTytj2FqAsaL8UuNx/6TbgO7Mpq6pPAE8AVFVV2WrUJmSvN3RRkp1Ciic6bx0tzU/nutIsntxzglTPu9+ONnWzmSuh1Pj3A5UiUi4iHiZv1m4PvkBEKoN2PwwcD2xvB+4TkSQRKQcqgbeuPmxjYGB0grdP97IsCtv3g927voSzfaPW3GPmzYw1flX1isjDwE7ADTypqjUi8ihQrarbgYdF5E5gAjgHPBAoWyMiTwNHAS/weVW1oYomLN460YPPryyNovb94EVjzhv3+nHJ5P2Ku9cUOhCViTchTWOoqjuAHVOOfTVo+4uXKfsN4BtXGqAxl7KnoYukBBeLclKdDuWqZCQnsmxhOodP9/LB1QVRM/rYRC8buWui1usNXWwozyHRHf3/xuvKsukdmeBU97DToZg4EP3vGBOXOvpHOdY+yC3L8pwOJSxWF2Xicbs4dPqc06GYOGCJ30Sl1xu7AHhvjCR+T4KLNcWZvHOmjwmf3+lwTIyzxG+i0p7j3SxITWR1UabToYTNurIFjE74qW8bcDoUE+Ms8Zuoo6q83tDFLUvzcLli50ZoRX46GUkJHDrd63QoJsZZ4jdRp7FziLb+0Zhp3z/P7RKuKc3iWPsAY7Yer5lDlvhN1Hm9Ibba94NdU5yF16/Ut1tzj5k7lvhN1NnT0EVZTgqLcqO7//50FuemkpGUwO/O9DkdiolhlvhNVJnw+dnb2B2TtX0AlwirizOpbx9geNzrdDgmRlniN1Hl7eZeBsa83FaZ73Qoc+aakiwmfMor9TZTrZkbIU3ZYEwk2LqvmeePtuESONs7Ou28N7FgSW4aaR43v3mnlXvWFjkdjolBVuM3UeV4+yBl2alROw1zKNwuYXVxFrvrOhi13j1mDljiN1FjcMzL2d4RKguiZzbOK7WmOJPhcR9vBEYoGxNOlvhN1GjoGESByoUZTocy5yry0khJdLO7rsPpUEwMssRvosbx9gFSPW5KslOcDmXOJbhd3LIsj5fqOlG1RelMeFniN1HB71eOdwyybGF63MxX/4GVCznTO8LxjkGnQzExxhK/iQq1bf0MjnnjopnnvPevnOyyas09JtxCSvwisklE6kWkQUQemeb8l0XkqIgcEZFdIrI46JxPRA4FfrZPLWtMKF442o4Ay+Pgxu55RVkprCzM4CVL/CbMZkz8IuIGHgfuAVYD94vI6imXvQ1Uqeq1wDPAY0HnRlR1XeBnc5jiNnFmZ007i3JTyUhOdDqUefWBlQupPnWOvpEJp0MxMSSUGv8GoEFVm1R1HNgGbAm+QFVfUtXza8btBUrDG6aJZ6d7hqlt7Y+pufdDsXVfMz6/4vMrf72jNmYHrJn5F0riLwFOB+23BI5dyueA54L2k0WkWkT2isi90xUQkQcD11R3dtowdfNuO2vaAFhTnOVwJPOvLCeVlEQ3x9vtBq8Jn1CmbJiuC8W0/ctE5I+BKuB9QYcXqepZEakAdovIO6ra+K4HU30CeAKgqqrK+q6Zd3m+pp2VhRnkpHmcDmXeuUSoyE+joXPQunWasAmlxt8ClAXtlwJnp14kIncC/xXYrKpj54+r6tnA7ybgZWD9VcRr4kzX4Bj7T/Vw95pCp0NxzNL8dPpGJugeGnc6FBMjQkn8+4FKESkXEQ9wH/Cu3jkish74LpNJvyPoeLaIJAW284BbgKPhCt7EvhePtqMKd60pcDoUxyzLn+zJ1NhpzT0mPGZM/KrqBR4GdgK1wNOqWiMij4rI+V46fwOkAz+f0m1zFVAtIoeBl4BvqqolfhOynTVtlGanxN2N3WC56R6yUhJptIFcJkxCmpZZVXcAO6Yc+2rQ9p2XKPcGsPZqAjTxaeu+ZkYnfLx6vIubynN46q3TMxeKUSLC0vx0alv78fs1phaYN86wkbsmYh1rH8DnV1bHYW+eqZbmpzEy4eNoa7/ToZgYYInfRKyjrf2kedwsjsG1dWdraaCd//xC88ZcDUv8JiJ5fX7q2wZYVZQZN5OyXU5mSiILM5J4vbHb6VBMDLDEbyJSY+cQY14/q4vj96buVEvz03nrRDfjXr/ToZgoZ4nfRKSjrX14ElwXmjgMVOSnMTrh50hLr9OhmChnid9EHJ9fOdo6wIqCDBLd9i963pLcNAD2nehxOBIT7exdZSLO283nGBrzWjPPFGlJCawszGBvk7Xzm6tjid9EnJ01bbhFWFEQP4uuhOqmilyqT55jwmft/ObKWeI3EUVV2VnTztKFaSQnup0OJ+JsLM9hZMLHkZY+p0MxUcwSv4kodW0DNPcMs7rIBm1NZ0N5DoA195irYonfRJSdNW2IwKoia+aZTm56EisKrJ3fXB1L/CaiPF/TzvWLsuNuicXZuKkihwOnrJ3fXDlL/CZinO4Z5mhrP3fH8RTModhYkcvwuI93zlg7v7kylvhNxDi/xGI8L7oSCmvnN1fLEr+JGOeXWFwcGKhkppeXnsTygnT2NtlALnNlQpqP35i5dn6JxT//QKXToUS0rfuaAchO9bC3qZsfv3kKt0v4o42LHI7MRJOQavwisklE6kWkQUQemeb8l0XkqIgcEZFdIrI46NwDInI88PNAOIM3sePCEourrX0/FBX56Yx7/ZztHXE6FBOFZkz8IuIGHgfuAVYD94vI6imXvQ1Uqeq1wDPAY4GyOcDXgI3ABuBrIpIdvvBNrHj+aDslC1JYY9M0hKQ8b7I5rKlryOFITDQKpca/AWhQ1SZVHQe2AVuCL1DVl1R1OLC7FygNbN8NvKCqPap6DngB2BSe0E2sGBzzsud4F3evKURs7v2QpCclkJ+RxIkuW4fXzF4oib8ECF7wtCVw7FI+Bzw3m7Ii8qCIVItIdWdnZwghmVjycn0H4z6/deOcpYq8NE52D+Pzq9OhmCgTys3d6apg0/6nicgfA1XA+2ZTVlWfAJ4AqKqqsv/iOLJ1XzPb9jeT6nFzvGOQxk5rughVeV4a+070WDu/mbVQavwtQFnQfilwdupFInIn8F+Bzao6NpuyJn75/Dq5xGKhLbE4W+fb+U9YO7+ZpVAS/36gUkTKRcQD3AdsD75ARNYD32Uy6XcEndoJ3CUi2YGbuncFjhkDwMnuySUWbW6e2ctITiQ/PYkma+c3szRj4ldVL/Awkwm7FnhaVWtE5FER2Ry47G+AdODnInJIRLYHyvYAX2fyw2M/8GjgmDEA1LX243YJSxfaEotXYunCNE52Dds6vGZWQhrApao7gB1Tjn01aPvOy5R9EnjySgM0sUtVqWsboCIvjaQEm3v/SizLnxzB+3bzOTZW5DodjokSNmWDcUxT1xDdQ+OsLLK++1eqPC8dAV5v6HI6FBNFLPEbx+yunbwdtLLQ2vevVIrHTWl2Cnss8ZtZsMRvHPNibTuFmclkp3qcDiWqLVuYzuGWPvpHJ5wOxUQJS/zGEX3DE1SfOscKq+1ftaUL0/H5lb2NNk2zCY0lfuOIl4914PMrqyzxX7VFOamkJLqtnd+EzBK/ccTuug5y0jyU5qQ6HUrUS3C52FiRw2uW+E2ILPGbeef1+Xm5vpP3r1hoo3XD5L3L8mjqHOKMTd9gQmCJ38y7A6fO0TcywR2rFjodSsy4fcXka7m7rmOGK42xxG8csLuug0S3cGtlntOhxIyl+Wkszk1lV22706GYKGCJ38y7F2vb2VieS0ZyotOhxAwR4Y6VBbzR2M3wuNfpcEyEs8Rv5s3Wfc38w67jNHYOkZWSeGH9WBMed65ayLjXz2vH7SavuTxL/GZe1bUNADZady7cWJ5DRnKCNfeYGVniN/Oqrq2f/IwkctOTnA4l5iS6XbxveT676zrx26pc5jIs8Zt5Mzrh40TXkA3amkN3riqga3CMwy29TodiIpglfjNvjncM4ldYUWizcc6V21fk43YJLxy15h5zaZb4zbypa+0nJdHNIhutO2cWpHp4z9Jcnj3Siqo195jphZT4RWSTiNSLSIOIPDLN+dtE5KCIeEXk41PO+QKrcl1YmcvEH59fqW8fYEVhBm6XjdYNt637mi/85Kcn0dwzzJGWPqfDMhFqxsQvIm7gceAeYDVwv4isnnJZM/BZYOs0DzGiqusCP5unOW/iwKHT5xge91lvnnmwpjgLtwi/PnzW6VBMhAqlxr8BaFDVJlUdB7YBW4IvUNWTqnoEsIU/zbR21XbgEqhcaIl/rqV43CwvSOfZI63Wu8dMK5TEXwKcDtpvCRwLVbKIVIvIXhG5d7oLROTBwDXVnZ2ds3hoEy121XawODeNFI+trTsfri1dQFv/KPtP9jgdiolAoST+6RpkZ1ONWKSqVcAfAX8nIksvejDVJ1S1SlWr8vPzZ/HQJhqc7hmmvn3AunHOo5VFGSQnuvj1EWvuMRcLJfG3AGVB+6VAyP9Nqno28LsJeBlYP4v4TAx4PtC1cJUtqj5vkhLc3LmqgN8caWXcay2w5t1CSfz7gUoRKRcRD3AfEFLvHBHJFpGkwHYecAtw9EqDNdHp+Zo2VhZm2GjdefaH15dybnjCpnAwF5kx8auqF3gY2AnUAk+rao2IPCoimwFE5EYRaQE+AXxXRGoCxVcB1SJyGHgJ+KaqWuKPI92DY+w/2cNdqwucDiXu3LY8n8LMZH5WfXrmi01cSQjlIlXdAeyYcuyrQdv7mWwCmlruDWDtVcZootiuug78CnetKbR+5fPM7RI+UVXK4y810No3QlFWitMhmQhhI3fNnHq+pp3irGTWFFv7/nzbuq+ZpAQ3foX//q81FwZ4GWOJ38yZ4XEvrx3v5K41hYitreuInDQPFflpHDjVg9+mcDABlvjNnHn1WBdjXj93rbH2fSdVLc7h3PAETZ1DTodiIkRIbfzGzNbWfc089VYzqR43jR1DnOwadjqkuLWmOJNUj5u9Td0sW5judDgmAliN38yJsQkfta39XFuaZZOyOSzR7eLGJTnUtvbTOzzudDgmAljiN3PiaGs/Xr9yXekCp0MxwIbyHAD2nbApHIwlfjNHDp3uJTs10ebejxDZqR5WFWWy/2QPoxM+p8MxDrPEb8Kuc2CMxs5Bri1dYL15IshNFbkMj/v4zZFWp0MxDrPEb8Juxzut+BXWlVkzTyRZmp9GfkYSP3zzpK3OFecs8Zuw++XbZyjMTKYgM9npUEwQEeHmilyOtPRx6LQtxh7PLPGbsPrdmT4On+7lhsXZTodiprG+bAHpSQn86M1TTodiHGSJ34TVj988RXKii+sXWeKPREmJbj5+Qym/OdJK58CY0+EYh1jiN2HTNzzBrw6f4d51JbbSVgT79M2LGff5+dl+m7cnXlniN2Hz8wOnGZ3w8+mbFzsdirmMpfnp3FqZx0/2NjPhs0Va4pElfhMWfr/y033NXL9oAWuKs5wOx8zgs+9ZQlv/KM/9rs3pUIwDLPGbsHj5WAcnuob4zM1LnA7FzGDrvmZa+0bJS0/iW8/V8dO9dqM33oSU+EVkk4jUi0iDiDwyzfnbROSgiHhF5ONTzj0gIscDPw+EK3ATOVSV/7urgdLsFD58bZHT4ZgQuES4ZVkuZ3pHONltE+jFmxkTv4i4gceBe4DVwP0isnrKZc3AZ4GtU8rmAF8DNgIbgK+JiHX3iDF7Gro4dLqX/3D7UhLd9iUyWqwvyybV42bP8U6nQzHzLJR36QagQVWbVHUc2AZsCb5AVU+q6hFg6p2iu4EXVLVHVc8BLwCbwhC3iSD/sLuBoqxkPn7DRatvmgjmSXCxsTyXurYBmjoHnQ7HzKNQEn8JELxac0vgWChCKisiD4pItYhUd3Za7SOa7Gvq5q0TPfz72ypISrAunNHmpooc3C7hB3tOOB2KmUehJP7pZtkKdaKPkMqq6hOqWqWqVfn5+SE+tHGaqvLtF46Rn5HEfRsWOR2OuQIZyYmsK1vAMwda6BmyufrjRSgrcLUAZUH7pcDZEB+/Bbh9StmXQyxrItxL9R3sO9HD5uuK+eXBM06HY67QLcvyqD51jp/sPcUX7qh0OhwzD0Kp8e8HKkWkXEQ8wH3A9hAffydwl4hkB27q3hU4ZqKcz69867l6ctM83Lgkx+lwzFUoyEzm9hX5/OjNkzZXf5yYMfGrqhd4mMmEXQs8rao1IvKoiGwGEJEbRaQF+ATwXRGpCZTtAb7O5IfHfuDRwDET5X55sIX69gHuWlNoSyvGgD+9tYKuwXF+dci+ucWDkBZbV9UdwI4px74atL2fyWac6co+CTx5FTGaCDM64ePbLxzjurIFXFOc6XQ4JgzeszSXVUWZfO+1E3zihjJc9mEe06zTtZm1H75xkta+UR7ZtNJW2IoRIsJD76ugoWOQ54+2Ox2OmWMh1fiNgcmh/iPjPv72xWMsL0jnRNeQ0yGZMNm6rxmfX8lN8/D1Z4/SPTjGv7nJJtuLVVbjN7PyyrEOxib83L2m0OlQTJi5XcL7ludzpneE4x02oCuWWeI3IesdHueNxm7WlS2gKCvF6XDMHFi3aAFZKYm8VNdh6/LGMEv8JmS7ajtQ4M7VBU6HYuZIgsvFbZV5nOoZ5s2mbqfDMXPEEr8JSX3bAAebz3FzRS7ZqR6nwzFzqGpJDpnJCfzvnfVW649RlvhNSB77bR1JiS5uX25TasS6RLeLD6ws4GBzL7vrOpwOx8wBS/xmRvuautlV18H7KvNJTbKOYPHghsXZLMlN5W921uP3W60/1ljiN5elqvz1c3UUZCZx89I8p8Mx88TtEr70weXUtQ3w6yOhTs1looUlfnNZz/2ujUOne/nSncvxJNi/Szz5g2uLWVWUyWO/rWdk3ObwiSX2TjaXNOHz89hv61hekG6LrMQhl0v42h+s5kzvCN95pdHpcEwYWeI3l/TUW82c7B7mrzatJMGWVIxLN1Xksvm6Yv75lUaabW3emGHvZnORrfua+Zc9J/jWc3WU56XR1jfK1n3NTodlHPJfPrSKBNxDAusAAAztSURBVJfw6LM1TodiwsQSv5nWq8e7GBr3sWlNoU3EFucKs5L54h2VvFjbwa8P243eWGB988xF+kcn2NPQydqSLMpyUp0Oxzgk+FteqieBsuwU/uKZw9y4JIfCrGQHIzNXy2r85iK7ajvw++Eum5rBBLhdwieqyvD5lb945rD17Y9yISV+EdkkIvUi0iAij0xzPklEfhY4v09ElgSOLxGRERE5FPj55/CGb8KtoWOAA6d62FCeQ256ktPhmAiSl57Eh9YW8drxLr6/p8npcMxVmLGpR0TcwOPAB5lcPH2/iGxX1aNBl30OOKeqy0TkPuBbwKcC5xpVdV2Y4zZzQFX5xm9qSXS7eP/KhU6HYyLQhiU5jE34+eZzdawszOQ2m8IjKoVS498ANKhqk6qOA9uALVOu2QL8MLD9DHCH2B3BqPP80XZequ/kjpULSbepGcw0RIT/88nrWF6QwcNbD9LUafP2R6NQEn8JcDpovyVwbNprAouz9wG5gXPlIvK2iLwiIrdeZbxmjgyPe3n010dZUZBhUzOYy0pLSuB7n6kiwe3i3/2wmp6hcadDMrMUSuKfruY+9c7Opa5pBRap6nrgy8BWEblodW4ReVBEqkWkurOzM4SQTLj94+4GzvSO8PV7r8FtC22by9i6r5nXjnfxiRtKae4ZZvM/7mFozOt0WGYWQkn8LUBZ0H4pMLUz74VrRCQByAJ6VHVMVbsBVPUA0Agsn/oEqvqEqlapalV+vrUZzrejZ/v53mtNfOz6EjaU5zgdjokSi3PTuH/DIs72jvDQTw4w7vU7HZIJUSiJfz9QKSLlIuIB7gO2T7lmO/BAYPvjwG5VVRHJD9wcRkQqgErAugNEkDGvjy8/fYgFqR7++4dXOx2OiTKrijL56PoSXjvexX/+uXXzjBYz3sFTVa+IPAzsBNzAk6paIyKPAtWquh34AfBjEWkAepj8cAC4DXhURLyAD3hIVXvm4g8xV+bvXjxOXdsAT362iuw0W1nLzN4Ni3MYGvOx/fBZOgfG+Mi1RRdGe//RxkUOR2emE1LXDVXdAeyYcuyrQdujwCemKfcL4BdXGaOZI/tP9vDdVxr5VFUZH1hpg7XMlbu1Mo/BMS97GrpIT07g/SusO3Aksz57capzYIyHtx5kQaqHFYUZNgmbuSoiwqZrChka8/LC0XbSPAl2vyiCWeKPQ16fnz9/6iB9IxP86a0VJCe6nQ7JxACXCB+7vpThcR+/OnSGVI/9X0Uqm6snDn3rt3XsberhG/eupSgrxelwTAxxu4T7NyyiLCeVn1Wf5s3GbqdDMtOwxB9Htu5r5j9uO8T3XjvBxvIcxqz7nZkDngQXn7l5MblpHv70R9X87kyf0yGZKSzxx5Fj7QNsP3yG5QXpfOTaYqfDMTEs1ZPAn9xSTlZKIp/9l7c41T3kdEgmiCX+OHGkpZen3mqmIDOZ+29cZKNzzZzLSknkk1VlDI/7+MPvvMEPXjthnQgihCX+OFDb2s+nf/AWqR43n7l5CUl2M9fMk/yMJP5442LODU/w032n8PqteTESWOKPcQ0dA3z6B/tISXTzufdWkJWS6HRIJs4syUvjY+tLaOoa4leHzqJqo3udZok/hh0+3csnv7sXEH76pxvJsZG5xiHrF2XzgZULOXDqHI/trHc6nLhniT9GvXqsk/u/t5e0JDc/f+hmluanOx2SiXN3rFzIhvIcvvNyI999pdHpcOKaDeCKMX6/8p1XGvn2C8dYXpDBD//tjSzMsIWxjfNEhM3XFbMwI4m/fq4Or1/5s9uXYms2zT9L/DGktW+Ev3zmCK8d7+IPrivmf330GjKSrU3fRA6XCN/+5DoSXMLf7KznZNcQ3/joWjwJ1vgwnyzxx4DRCR/fe7WJf3q5Eb8qf/2xtdx3Y5nVpExE8iS4+NtPrWNxbhp/v2tydtj/uWUN1y/Kdjq0uGGJP4r1DU/wl88c5s2mbvpHvawpzuRD1xShCk+9dXrmBzDGISLClz64nBWFGfyP7TV87J/e4N51xdy/YRE3LsnBZeNM5pQl/igz7vWzp6GTf337LC8cbWdkwseyhel8siqfCruBa6LMh9YWcdvyfP5xdwM/fOMk/3roLEVZybx3WR7rF2Vz/eIFVC7MsAGHYSaR1qe2qqpKq6urnQ4jYvj8Sn3bAAdO9fDq8S7eaOhiaNzHgtREPrS2iNw0j020ZmLCuNdPbWs/75zp42T3EMPjPgCSElyUZqewKCeVspxUvnTncls0aBoickBVq0K61hJ/ZBid8HG6Z5jmwE9DxyB1bQPUtfYzFHgDLEhNZHlBBisLMlhWkE6Cy26ImdikqvQMjV94P5zuGaatf5TzKztW5KWxflE271may3uW5Vrlh9kl/pCaekRkE/D3TC69+H1V/eaU80nAj4AbgG7gU6p6MnDuK8DnmFx68QuqujPEvyNqjXv9DI15GRzzMjTuDWz7GBrzMjA6QUf/GO0Doxw81Uv/6AR9IxMMjHrf9RiZyQmsLMzkD28o5fpF2bScGyE7NdFu2Jq4ICLkpieRm57E+sBN33Gvn5beYXLTkjjYfI7dde384mALMPlBcPPSXG5cksOKwgyW5qdbT6HLmLHGH1gs/RjwQaCFycXX71fVo0HX/Blwrao+JCL3AR9V1U+JyGrgKWADUAy8CCxXVd+lnm8uavyqil/Br4rPr2hg2x84rqp4/YrXp0z4/IFtPxO+yetHvT4GRifoH5lM3P2jXvpGJjhw6hwj4z5GJnyMjPsYHvcyMuFjwjfzt6js1EQ8CS4ykxPJTE4kJ91DTqqHnDQP2Wke0jxuS/LGXIZflfb+URo7h2jsGKTl3PCFb8cumZwnqDAzmaxUDymJLlI9CSQnuklJdJOU6MLjduFJcJGU8O7fHrd78nfQsUSXC7dLSHALbpdM7ruFBJfgEsElkx9Wwb8njwtyYZsL+3Px3g53jX8D0KCqTYEH3wZsAY4GXbMF+B+B7WeAf5TJv2wLsE1Vx4ATgcXYNwBvhhLcbHQPjnHrYy+9K5n7/Hrhq2G4JQX+KVI8blISE8hO81CyIIUUj5vkRPeF80lB254EF8mJbtKTEkh0W23EmKvhEqEoK4WirBTeuywPn1/pHBijvX+UjoFR+ka89I9O0D00yITPz7h3sjI37vPj8yk+h5u5p/twuK4si20P3jznzx1K4i8BgvsGtgAbL3WNqnpFpA/IDRzfO6VsydQnEJEHgQcDu4MiYpN5QB7Q5XQQEcBeh9+z1+L3YvK1qAN+9u9nXez8a7E41AKhJP7pvpNM/ai81DWhlEVVnwCeCCGWuCEi1aF+bYtl9jr8nr0Wv2evxe9dyWsRSntDC1AWtF8KnL3UNSKSAGQBPSGWNcYYM49CSfz7gUoRKRcRD3AfsH3KNduBBwLbHwd26+Rd4+3AfSKSJCLlQCXwVnhCN8YYcyVmbOoJtNk/DOxksjvnk6paIyKPAtWquh34AfDjwM3bHiY/HAhc9zSTN4K9wOcv16PHvIs1fU2y1+H37LX4PXstfm/Wr0XEDeAyxhgzt6xPoTHGxBlL/MYYE2cs8UcQESkTkZdEpFZEakTki07H5DQRcYvI2yLyrNOxOElEFojIMyJSF/j/mPtRPhFKRL4UeH/8TkSeEpG4WWJORJ4UkQ4R+V3QsRwReUFEjgd+z7iwgSX+yOIF/pOqrgJuAj4fmPYinn0RqHU6iAjw98BvVXUlcB1x+pqISAnwBaBKVa9hssPJfc5GNa/+H7BpyrFHgF2qWgnsCuxfliX+CKKqrap6MLA9wOSb+6KRzvFCREqBDwPfdzoWJ4lIJnAbk73nUNVxVe11NipHJQApgTFDqcTR2CBVfZXJnpPBtgA/DGz/ELh3psexxB+hRGQJsB7Y52wkjvo74C8Bv9OBOKwC6AT+JdDs9X0RSXM6KCeo6hngfwPNQCvQp6rPOxuV4wpUtRUmK4/AwpkKWOKPQCKSDvwC+I+q2u90PE4QkY8AHap6wOlYIkACcD3wHVVdDwwRwtf5WBRov94ClDM542+aiPyxs1FFH0v8EUZEEplM+j9V1V86HY+DbgE2i8hJYBvwARH5ibMhOaYFaFHV89/+nmHygyAe3QmcUNVOVZ0Afgm8x+GYnNYuIkUAgd8dMxWwxB9BAlNZ/wCoVdVvOx2Pk1T1K6paqqpLmLx5t1tV47Jmp6ptwGkRWRE4dAfvnhY9njQDN4lIauD9cgdxeqM7SPCUOQ8Av5qpgC22HlluAT4NvCMihwLH/ouq7nAwJhMZ/hz4aWC+rCbgTxyOxxGquk9EngEOMtkL7m3iaPoGEXkKuB3IE5EW4GvAN4GnReRzTH4wfmLGx7EpG4wxJr5YU48xxsQZS/zGGBNnLPEbY0ycscRvjDFxxhK/McbEGUv8xhgTZyzxG2NMnPn/ONNJmb++AxcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "s1 = np.random.normal(5, 1, 10000)\n",
    "sns.distplot(s1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a20cb6be0>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD4CAYAAADSIzzWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAANYElEQVR4nO3df2jc9R3H8dc7P4ZpM9FGiV2j3sqBMiaoDe6HUHDqaOLYBv7jYGv+mPiHI0b3R1gxEANhgk5ZzR8DcZsNE2VT949Liso6nLDNJc6uOkO4bdemaVrbiD9S7dpr3vvjLmdyTfWul7v3NXk+QLye3+993sbcs9/75Fs0dxcAoPrqogcAgLWKAANAEAIMAEEIMAAEIcAAEKShlIMvueQSTyQSFRoFAFan8fHxY+5+aeHzJQU4kUhobGxs5aYCgDXAzPYv9zxbEAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEFK+n/CAeUaGhpSKpWq2nrT09OSpE2bNlVlvWQyqe7u7qqshfMfAUZVpVIpvfHm2zq9bkNV1qv/6H1J0uH/Vf5bvf6jdyu+BlYXAoyqO71ugz6+urMqazVNjEhSVdZbWAsoFnvAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAl2BoaEhDQ0PRYwCr1lp7jzVED3A+SaVS0SMAq9pae49xBQwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAARpqMYis7OzGhgYUH9/v1paWso6L5VKqaenRzt37tR7772n3t5ePfzww9qyZcsZx8/OzmrHjh3av3+/zExXXHGFent79dBDD+nAgQOan59XXV2dLrvsMtXV1WlmZkYtLS2amZnR6dOntX37dm3dulV33323Tp48KUnavHlzRb5GALJOnTqlO++8U9PT09qwYYOmp6e1ceNGzczMyMzU1tampqYmZTIZHTp0SO6uK6+8Ur29vXrkkUd04sQJHT58WJdffrkefPBBSdLAwIDuuecePfbYY+rv75ck7dixQ1NTUxoaGlIymVzSj4VzCh+X0q9iVCXAu3bt0r59+zQ8PKz77ruvrPMGBwd1/PhxDQ4O6tixY5qfn1d/f79eeOGFM47ftWuXJicn8683OTmpwcFBpdPpJess/vXBgwfzj4eHh/XKK6/k41t4LICVd+TIEc3OzkqSpqenJUkzMzOSJHfX1NTUGecs996enJzU8PCw3F379u3T4OCg9u/fn39uoQ2Dg4N68sknl/Rj4ZzCx6X0qxgV34KYnZ3V7t275e7avXt3/gt7LuelUqn8FzidTmtubk6SNDc3pz179iw5PpVKaXR09IzXLTWghcfPz89rfHy8pNcAUJxTp04V3YhCy723R0ZGNDo6KndXOp2Wu2t0dFQjIyNLzhsbG8v3Y3R0NH/O4sel9KtY5u5FH9ze3u5jY2MlLfDoo49qZGREmUxGDQ0Nuu2224r6XWS58/bu3XvWgDY0ZC/mF45va2ur2NVqXV2drrnmmoq89mqXSqX04UnX8WvvqMp6TRPZN9rHV3dWfK31bzyjz3/OlEwmK77WavXWW28pk8ms6GuamRZ3rvDXktTc3KwTJ04ok8nIzCRlr7YXPy6lX8vMMO7u7YXPf+YVsJndZWZjZjZ29OjRkhd++eWX81/QTCajl1566ZzP+7SgZjKZJcdXcqtgfn6+Yq8NrGUrHV9JZ8R2uYvOubm5/Nrunj9m8eNS+lWsz9wDdvfHJT0uZa+AS13glltuWXIle+utt57zebVyBdzc3KydO3dW5LVXu56eHo3/50j0GBUxf8GFSm5u5XujDLfffvvKf8xfwSvgYvtVrIrvAXd1damuLrtMfX29tm/ffs7n9fX1nfX4+++/f8nxfX19amxsLHP65Q0MDFTkdYG1rrW1dUVfr7GxMX9x9mnPPfDAA/l+LP7nix+X0q9iVTzALS0t2rZtm8xM27ZtK/o2juXOSyaTSiQSkqREIqHm5mZJ2d+9brrppiXHJ5NJdXR0nPG6C+cXq/D4uro6bdmypaTXAFCcxsbGc77Va7n3dmdnpzo6OmRmSiQSMjN1dHSos7NzyXnt7e35fnR0dOTPWfy4lH4Vqyq3oXV1dSmdTpf8u8dy5/X19amnp0d9fX35+4AXrkgLj+/q6tLExMSK3gdcasABlKa1tVUXX3zxitwHvNCCdDqdvw944bmJiQlNTU3lP1kX9uNsj1dSxe+CWE16enokiT2+MizsAVfjrgSpundBNE2MaAt7wGVZre+xc74LAgBQGQQYAIIQYAAIQoABIAgBBoAgBBgAghBgAAhCgAEgCAEGgCAEGACCEGAACEKAASAIAQaAIAQYAIIQYAAIQoABIAgBBoAgBBgAghBgAAhCgAEgCAEGgCAEGACCEGAACEKAASAIAQaAIAQYAIIQYAAIQoABIAgBBoAgBBgAghBgAAhCgAEgCAEGgCAEGACCEGAACEKAASAIAQaAIAQYAII0RA9wPkkmk9EjAKvaWnuPEeASdHd3R48ArGpr7T3GFgQABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEaogfA2lP/0btqmhip0lqzklSV9eo/eldSa8XXwepBgFFVyWSyqutNT2ckSZs2VSOMrVX/98P5jQCjqrq7u6NHAGoGe8AAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABCHAABCEAANAEAIMAEEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQhAADQBACDABBCDAABDF3L/5gs6OS9pe4xiWSjpV4TjUxX/lqfUbmK1+tz1jr813p7pcWPllSgM+FmY25e3tFFykD85Wv1mdkvvLV+oy1Pt/ZsAUBAEEIMAAEqUaAH6/CGuVgvvLV+ozMV75an7HW51tWxfeAAQDLYwsCAIIQYAAIUrEAm9mvzOwdM3uzUmuUw8wuN7M9Zva2mb1lZj3RMy1mZheY2Wtmtjc330D0TMsxs3oz+4eZvRA9y3LMLG1m+8zsDTMbi56nkJldZGbPmtlE7nvxa9EzLTCzq3Jft4W/PjCze6PnKmRm9+XeI2+a2dNmdkH0TMWq2B6wmW2VNCdp2N2/XJFFymBmGyVtdPfXzezzksYlfdfd/xU8miTJzEzSenefM7NGSa9K6nH3vwaPtoSZ/VhSu6QL3f1b0fMUMrO0pHZ3r8mb9M1sl6Q/u/sTZvY5Sevc/b3ouQqZWb2kaUlfcfdS/zBWxZjZJmXfG19y94/N7LeSRtz9ydjJilOxK2B3f0XSu5V6/XK5+4y7v557/KGktyVtip3qE541l/tlY+6vmvqJqZm1SbpN0hPRs5yPzOxCSVsl/VKS3P1kLcY352ZJ/66l+C7SIKnJzBokrZN0KHieorEHLMnMEpKuk/S32EmWyn28f0PSO5Jecveamk/SzyX1SpqPHuRTuKQXzWzczO6KHqbAZklHJf06t43zhJmtjx7qLO6Q9HT0EIXcfVrSzyQdkDQj6X13fzF2quKt+QCbWbOk5yTd6+4fRM+zmLufdvdrJbVJusHMamYrx8y+Jekddx+PnuUz3Oju10vqkPSj3NZYrWiQdL2kX7j7dZKOS/pJ7Ehnym2NfFvS76JnKWRmF0v6jqQvSvqCpPVm9v3YqYq3pgOc21t9TtJT7v589Dxnk/tY+idJ24JHWexGSd/O7bE+I+kbZvab2JHO5O6Hcn9/R9LvJd0QO9ESByUdXPTJ5lllg1xrOiS97u5HogdZxi2S/uvuR939lKTnJX09eKairdkA537I9UtJb7v7o9HzFDKzS83sotzjJmW/0SZip/qEu+9w9zZ3Tyj78fSP7l5TVx5mtj73A1blPtp/U1LN3JXj7oclTZnZVbmnbpZUEz8ELvA91eD2Q84BSV81s3W59/TNyv4857xQydvQnpb0F0lXmdlBM/thpdY6RzdK+oGyV24Lt9l0Rg+1yEZJe8zsn5L+ruwecE3e6lXDWiW9amZ7Jb0m6Q/uvjt4pkLdkp7K/Xe+VtJPg+dZwszWSbpV2SvLmpP79PCspNcl7VO2aefNH0vmjyIDQJA1uwUBANEIMAAEIcAAEIQAA0AQAgwAQQgwAAQhwAAQ5P9bmJwrREAQuQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# for boxplots see https://en.wikipedia.org/wiki/Interquartile_range (or ask!)\n",
    "sns.boxplot(s1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Heavy-tailed\n",
    "Distributions with a small but non-negligible amount of observations with high values. Several probability distributions follow this pattern: https://en.wikipedia.org/wiki/Heavy-tailed_distribution#Common_heavy-tailed_distributions.\n",
    "\n",
    "We pick the lognormal here: https://en.wikipedia.org/wiki/Log-normal_distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a20d80860>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAD5CAYAAADm8QjUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dfZBd9X3n+ffn3tsPklrosQGBhCUb+UE4XmzLsl3xTnkhDrInYzm1uGgy8VC7bDEzCzVJJTsxTKo8CRWmTGomZHYW22ECCWaMBcGZocvDDGsbp5zsxIIGyxhBZDo8iLZkJNCzhLrvw3f/OL9unb66t/v0k9SiP6+qrj733N/53d9pLvej38M5VxGBmZnZdJTOdQPMzOz85RAxM7Npc4iYmdm0OUTMzGzaHCJmZjZtDhEzM5u2SpFCkrYC/x4oA38aEV9uer4L+DrwYeBN4LqIeCU9dxtwI1AH/kVEPJ47rgwMAD+LiF9J+zYA24GVwDPAFyJiZKL2rV69OtavX1/kVMzMLHn66affiIjemdQxaYikD/q7gU8BQ8BTkvoj4vlcsRuBQxFxuaQ+4E7gOkmbgD7gCuAS4LuS3h0R9XTcbwAvABfk6roTuCsitkv6Wqr7qxO1cf369QwMDBQ4XTMzGyXp1ZnWUWQ4awswGBEvpR7BdmBbU5ltwP1p+xHgaklK+7dHxHBEvAwMpvqQtBb4h8CfjlaSjrkq1UGq83PTOTEzM5t7RULkUuC13OOhtK9lmYioAUeAVZMc+8fA7wCN3POrgMOpjnavZWZm80SREFGLfc33SmlXpuV+Sb8C7I+Ip6fxWllB6SZJA5IGDhw40KqImZnNsSIhMgSsyz1eC+xtV0ZSBVgGHJzg2F8EPivpFbLhsask/SfgDWB5qqPdawEQEfdExOaI2NzbO6N5ITMzm6YiIfIUsFHSBkmdZBPl/U1l+oEb0va1wBOR3dmxH+iT1JVWXW0EnoyI2yJibUSsT/U9ERG/no75fqqDVOejMzg/MzObQ5OGSJqfuAV4nGwl1cMRsUvS7ZI+m4rdC6ySNAj8FnBrOnYX8DDwPPDfgZtzK7Pa+SLwW6muValuMzObh/R2uBX85s2bw0t8zcymRtLTEbF5JnX4inUzM5s2h0gbf/v3b/KLX36CE8O1yQubmS1QDpE2Bvcf42eH3+LAseFz3RQzs3nLIdLGcC27BvLkyGTrAMzMFi6HSBsj9SxE3qp6OMvMrB2HSBvVWrZqzT0RM7P2HCJtjNSz8HCImJm15xBpY2RsTsTDWWZm7ThE2qjWPZxlZjYZh0gbo6uz3nKImJm15RBpY8RLfM3MJuUQaaNad4iYmU3GIdLGyNhwlifWzczacYi0MeKeiJnZpBwibYwNZ1UdImZm7ThE2vDqLDOzyTlE2hidE/Gt4M3M2nOItFEduwGjeyJmZu0UChFJWyXtljQo6dYWz3dJeig9v0PS+txzt6X9uyVdk/Z1S3pS0o8l7ZL0+7nyfy7pZUk708+VMz/NqfN1ImZmk6tMVkBSGbgb+BQwBDwlqT8ins8VuxE4FBGXS+oD7gSuk7QJ6AOuAC4Bvivp3cAwcFVEHJfUAfyNpP8WET9M9f3LiHhktk5yOsZuBe8QMTNrq0hPZAswGBEvRcQIsB3Y1lRmG3B/2n4EuFqS0v7tETEcES8Dg8CWyBxP5TvST8zwXGbFgzv28OCOPRw+WQXg0MkRHtyx5xy3ysxsfioSIpcCr+UeD6V9LctERA04Aqya6FhJZUk7gf3AdyJiR67cHZKelXSXpK4pnM+sqTWyTBsd1jIzszMVCRG12Nfca2hXpu2xEVGPiCuBtcAWSe9Pz98GvBf4CLAS+GLLRkk3SRqQNHDgwIHJz2KK6o0sPGqNoBHzopNkZjbvFAmRIWBd7vFaYG+7MpIqwDLgYJFjI+Iw8FfA1vR4XxruGgb+jGw47QwRcU9EbI6Izb29vQVOY2rqjRhLwKp7I2ZmLRUJkaeAjZI2SOokmyjvbyrTD9yQtq8FnoiISPv70uqtDcBG4ElJvZKWA0haBPwS8Hfp8Zr0W8DngOdmcoLTVasHizrLwOlJdjMzG2/S1VkRUZN0C/A4UAbui4hdkm4HBiKiH7gXeEDSIFkPpC8du0vSw8DzQA24OSLqKSjuTyu/SsDDEfHt9JLfkNRLNhS2E/hns3nCRdQbQQCLOsqcHKl7XsTMrI1JQwQgIh4DHmva96Xc9ing822OvQO4o2nfs8AH25S/qkib5lI9Taov7izz5gn3RMzM2vEV6y2MhsjYcJZ7ImZmLTlEWqillVmLOjwnYmY2EYdIC6d7Itlon1dnmZm15hBpYfRCQ/dEzMwm5hBpoZabWAcYqfliQzOzVhwiLdSbQ8Q9ETOzlhwiLdTrTRPrnhMxM2vJIdLC6HBWR6VEWRr7giozMxvPIdLC6HBWpSQ6Khr7vnUzMxvPIdJCbSxESnSWS17ia2bWhkOkhdEQKZdEZ6XkiXUzszYcIi3kh7M6yyVPrJuZteEQaWH0C6nKZdHhnoiZWVsOkRbGDWeVS16dZWbWhkOkhXHDWRUPZ5mZteMQaaFWH786y8NZZmatOURaOGN1lnsiZmYtOURaqDcCASXh1VlmZhMoFCKStkraLWlQ0q0tnu+S9FB6foek9bnnbkv7d0u6Ju3rlvSkpB9L2iXp93PlN6Q6Xkx1ds78NKem3mhQLgkpW51Va8TYPImZmZ02aYhIKgN3A58GNgHXS9rUVOxG4FBEXA7cBdyZjt0E9AFXAFuBr6T6hoGrIuJ/Aq4Etkr6WKrrTuCuiNgIHEp1n1W1RlAuCch6IgBvVetnuxlmZvNekZ7IFmAwIl6KiBFgO7Ctqcw24P60/QhwtSSl/dsjYjgiXgYGgS2ROZ7Kd6SfSMdcleog1fm5aZ7btNUbQWU0RCrZn+jkSO1sN8PMbN4rEiKXAq/lHg+lfS3LREQNOAKsmuhYSWVJO4H9wHciYkc65nCqo91rkY6/SdKApIEDBw4UOI3iao2gknogYz2REfdEzMyaFQkRtdjXPEHQrkzbYyOiHhFXAmuBLZLeX/C1SMffExGbI2Jzb29v28ZPRz03nNUx1hNxiJiZNSsSIkPAutzjtcDedmUkVYBlwMEix0bEYeCvyOZM3gCWpzravdacazUn4hAxMztTkRB5CtiYVk11kk2U9zeV6QduSNvXAk9ERKT9fWn11gZgI/CkpF5JywEkLQJ+Cfi7dMz3Ux2kOh+d/ulNT73eOGNOxMNZZmZnqkxWICJqkm4BHgfKwH0RsUvS7cBARPQD9wIPSBok64H0pWN3SXoYeB6oATdHRF3SGuD+tFKrBDwcEd9OL/lFYLukPwB+lOo+q8b1RDyxbmbW1qQhAhARjwGPNe37Um77FPD5NsfeAdzRtO9Z4INtyr9EtiLsnBm3OsvDWWZmbfmK9RZqjaBSSquzPLFuZtaWQ6SFesuJdQ9nmZk1c4i0MH6Jb/bbE+tmZmdyiLRQazSolLPwqJRKlEviuHsiZmZncIi0UGsEZZ2+7rG7UuL4KYeImVkzh0gL9UaM9UQAujrKHHOImJmdwSHSQq0elEun/zTdHSWOnaqewxaZmc1PDpEW8teJAHRXyhwfdk/EzKyZQ6SF/Oos8HCWmVk7DpEmjQjq0dwTKTlEzMxacIg0Gf0a3DN7Ip4TMTNr5hBpMhoi43oiHSWOD9fIbjJsZmajHCJNaqM9kXJudValTCN8/ywzs2YOkSZjPRHlh7OyP5PnRczMxnOINBmbEynnh7PKAJ4XMTNr4hBpUqs3AM64TgTgmK8VMTMbxyHSpNZidVa3h7PMzFoqFCKStkraLWlQ0q0tnu+S9FB6foek9bnnbkv7d0u6Ju1bJ+n7kl6QtEvSb+TK/56kn0namX4+M/PTLK7V6qwuD2eZmbU06dfjpu9Bvxv4FDAEPCWpPyKezxW7ETgUEZdL6gPuBK6TtIns+9avAC4Bvivp3WTft/7bEfGMpKXA05K+k6vzroj4t7N1klNxuieSX52VbftOvmZm4xXpiWwBBiPipYgYAbYD25rKbAPuT9uPAFdLUtq/PSKGI+JlYBDYEhH7IuIZgIg4BrwAXDrz05m5Vhcbnp5Yd4iYmeUVCZFLgddyj4c48wN/rExE1IAjwKoix6ahrw8CO3K7b5H0rKT7JK0o0MZZU2+cObHeWSkheTjLzKxZkRBRi33Nl263KzPhsZJ6gG8BvxkRR9PurwLvAq4E9gH/rmWjpJskDUgaOHDgwMRnMAWtJtZLEj2dFa/OMjNrUiREhoB1ucdrgb3tykiqAMuAgxMdK6mDLEC+ERF/OVogIl6PiHpENID/SDacdoaIuCciNkfE5t7e3gKnUUytxcQ6QE93xcNZZmZNioTIU8BGSRskdZJNlPc3lekHbkjb1wJPRHajqX6gL63e2gBsBJ5M8yX3Ai9ExB/lK5K0JvfwV4HnpnpSMzG2Oqs8/k+ztLviiXUzsyaTrs6KiJqkW4DHgTJwX0TsknQ7MBAR/WSB8ICkQbIeSF86dpekh4HnyVZk3RwRdUmfAL4A/ETSzvRS/yoiHgP+UNKVZMNerwD/dBbPd1L1+pnDWQBLuzs4Nuw5ETOzvElDBCB9uD/WtO9Lue1TwOfbHHsHcEfTvr+h9XwJEfGFIm2aK7U0sd4cIj1dFQ6dHDkXTTIzm7d8xXqTdnMiHs4yMzuTQ6RJq+tEIBvOOuoQMTMbxyHSpNUSX8h6Ir5OxMxsPIdIk3ojKEuU1BQiXRWGaw1Gao1z1DIzs/nHIdKk3ogzeiGQ9UQAjvuCQzOzMQ6RJrVGo2WI9HR3AL71iZlZnkOkSa0eVMrteyK+at3M7DSHSJPJhrMcImZmpzlEmtQaccY1IgBLuzycZWbWzCHSpN4IKqUz/yyeWDczO5NDpEm74aweD2eZmZ3BIdKk3eos90TMzM7kEGnSbk6kq1Kms1LiqOdEzMzGOESatBvOguyqdQ9nmZmd5hBpMmGI+E6+ZmbjOESaTBwiHV7ia2aW4xBpMlGI9Hg4y8xsHIdIk3pkd/FtZWl3xauzzMxyCoWIpK2SdksalHRri+e7JD2Unt8haX3uudvS/t2Srkn71kn6vqQXJO2S9Bu58islfUfSi+n3ipmfZnGNSYezHCJmZqMmDRFJZeBu4NPAJuB6SZuait0IHIqIy4G7gDvTsZuAPuAKYCvwlVRfDfjtiHgf8DHg5lydtwLfi4iNwPfS47OmNkGILF/c4e9ZNzPLKdIT2QIMRsRLETECbAe2NZXZBtyfth8BrpaktH97RAxHxMvAILAlIvZFxDMAEXEMeAG4tEVd9wOfm96pTU8jglKbEFm5pJOTI3VOVetns0lmZvNWkRC5FHgt93iI0x/4Z5SJiBpwBFhV5Ng09PVBYEfadVFE7Et17QMuLNDGWVNvBJU2cyKrlnQC8OYJ90bMzKBYiLT6RI2CZSY8VlIP8C3gNyPiaIG2nH5B6SZJA5IGDhw4MJVDJzTR6qyVKUQOHneImJlBsRAZAtblHq8F9rYrI6kCLAMOTnSspA6yAPlGRPxlrszrktakMmuA/a0aFRH3RMTmiNjc29tb4DQmFxE0grbDWat6shB548TwrLyemdn5rkiIPAVslLRBUifZRHl/U5l+4Ia0fS3wRERE2t+XVm9tADYCT6b5knuBFyLijyao6wbg0ame1HRV61knqdW9sx7csYf/MfgmAP/12X08uGMPD+7Yc7aaZmY2L1UmKxARNUm3AI8DZeC+iNgl6XZgICL6yQLhAUmDZD2QvnTsLkkPA8+Trci6OSLqkj4BfAH4iaSd6aX+VUQ8BnwZeFjSjcAe4POzecITqdYbAJTazIks6cr+XCd8rYiZGVAgRADSh/tjTfu+lNs+RZsP+4i4A7ijad/f0Hq+hIh4E7i6SLtmWy31RNrNiXRVSpQlTgx7dZaZGfiK9XFGUk+kXYhIYklXmRMj7omYmYFDZJxaY+IQgWxIy8NZZmYZh0hOtZaGs9rMiQAs6XSImJmNcojkVAv1RMqcGPGciJkZOETGGVud5eEsM7NCHCI5o8NZra4TGbWkq8JwrUEtBY6Z2ULmEMkZHc5qd50IZHMigIe0zMxwiIxTrRWbEwFfcGhmBg6RcWqNiS82hFxPxCFiZuYQyZvsYkPI3frEFxyamTlE8ia77QlATwqR4771iZmZQyRvdInvRBcbdneUKMnDWWZm4BAZp1pgOEuSr1o3M0scIjnVAsNZkC449BJfMzOHSF6tQE8EYHFX2T0RMzMcIuOc/lKqict5OMvMLOMQyTn99bgT/1my4SyHiJmZQyTn9A0YJy7X01XmVLUx9v0jZmYLVaEQkbRV0m5Jg5JubfF8l6SH0vM7JK3PPXdb2r9b0jW5/fdJ2i/puaa6fk/SzyTtTD+fmf7pTU2R1Vlw+oLDk75WxMwWuElDRFIZuBv4NLAJuF7SpqZiNwKHIuJy4C7gznTsJqAPuALYCnwl1Qfw52lfK3dFxJXp57E2ZWbd2OqsCa4TgfxNGD2kZWYLW5GeyBZgMCJeiogRYDuwranMNuD+tP0IcLUkpf3bI2I4Il4GBlN9RMQPgIOzcA6zplpvUFJ2LchExm594p6ImS1wRULkUuC13OOhtK9lmYioAUeAVQWPbeUWSc+mIa8VBcrPilojJh3KAljanYXI0VPVuW6Smdm8ViREWn2qRsEyRY5t9lXgXcCVwD7g37VslHSTpAFJAwcOHJikymJGao1CIbJsUQcAR95yiJjZwlYkRIaAdbnHa4G97cpIqgDLyIaqihw7TkS8HhH1iGgA/5E0/NWi3D0RsTkiNvf29hY4jcnVGo0Jv5BqVEe5xOLOMkdOOkTMbGErEiJPARslbZDUSTZR3t9Uph+4IW1fCzwREZH296XVWxuAjcCTE72YpDW5h78KPNeu7Gyr1mLCr8bNW76owz0RM1vwKpMViIiapFuAx4EycF9E7JJ0OzAQEf3AvcADkgbJeiB96dhdkh4GngdqwM0RUQeQ9E3gk8BqSUPAv46Ie4E/lHQl2bDXK8A/nc0Tnki1UWw4C7IhrUPuiZjZAjdpiACkZbaPNe37Um77FPD5NsfeAdzRYv/1bcp/oUib5kK1HoWGswAuWNTBK2+enOMWmZnNb75iPadWL94TWb6og7eqdU76WhEzW8AcIjnVKYTIssXZCq29h0/NZZPMzOY1h0hOtV7sOhGAZYs6Adh35K25bJKZ2bzmEMmp1huT3vJk1Oi1IvvcEzGzBcwhklObQk/kgkXZmoS97omY2QLmEMkZmcKcSKVUoqer4p6ImS1oDpGcqUysQzakte+oQ8TMFi6HSE5tCteJQAqRwx7OMrOFyyGSM+WeyOIO9h1xT8TMFi6HSM5UbnsC2QWHx4drviW8mS1YDpGcaq346izwMl8zM4dITq1R/DoROB0iXuZrZguVQySn6JdSjRoNkZ97XsTMFiiHSE7Rr8cdtbS7g5LwCi0zW7AcIjlTXZ1VLokLl3az1z0RM1ugHCJJREzpBoyjLl2xiKFD/l4RM1uYHCJJrREAU7rYEOCylYt57aCHs8xsYXKIJLV6FiJFv2N91LqVi9l35C1Gao25aJaZ2bxWKEQkbZW0W9KgpFtbPN8l6aH0/A5J63PP3Zb275Z0TW7/fZL2S3quqa6Vkr4j6cX0e8X0T6+4kXoWAqUphshlKxfTCNjryXUzW4AmDRFJZeBu4NPAJuB6SZuait0IHIqIy4G7gDvTsZuAPuAKYCvwlVQfwJ+nfc1uBb4XERuB76XHc66aQmSqcyKXrVwMwJ6Dnhcxs4WnSE9kCzAYES9FxAiwHdjWVGYbcH/afgS4WpLS/u0RMRwRLwODqT4i4gfAwRavl6/rfuBzUzifaRsdzprKxYZwOkRedYiY2QJUJEQuBV7LPR5K+1qWiYgacARYVfDYZhdFxL5U1z7gwlaFJN0kaUDSwIEDBwqcxsSm2xO5cGkXnZUSrzlEzGwBKhIirT5Vo2CZIsdOS0TcExGbI2Jzb2/vjOs7HSJTO65UEutWLGLPmw4RM1t4inxkDgHrco/XAnvblZFUAZaRDVUVObbZ65LWpLrWAPsLtHHGqqPDWaWpL1h7x6olnhMxswWpyCfmU8BGSRskdZJNlPc3lekHbkjb1wJPRESk/X1p9dYGYCPw5CSvl6/rBuDRAm2csbGeyNRGs4DRa0VOkp2ymdnCMWmIpDmOW4DHgReAhyNil6TbJX02FbsXWCVpEPgt0oqqiNgFPAw8D/x34OaIqANI+ibwt8B7JA1JujHV9WXgU5JeBD6VHs+56c6JQHatyLHhGodP+ntFzGxhqRQpFBGPAY817ftSbvsU8Pk2x94B3NFi//Vtyr8JXF2kXbNp7Ir1aYRIfoXWiiWds9ouM7P5rFCILATV2vR6Ig/u2MPPj2Y3YPzmk3t4fu9RAH7to5fNbgPNzOYh3/YkqaaeSGWK14kArFyc9T4OnRiZ1TaZmc13DpFktCcyneGszkqJnq4KBx0iZrbAOESSWmP6E+sAK5d0OkTMbMFxiCQj07ztyaiVSzo5eNIhYmYLi0Mkqc1giS9kIXLkZHWsR2NmthA4RJKZXCcC2T20Ath72F+Va2YLh0MkGRvOmmaIXN7bg4AXXz82i60yM5vfHCLJ2HDWNOdEFndVuHTFIn7qEDGzBcQhksx0OAvg3RctZejQW5wcqc1Ws8zM5jWHSDJ6F9/pXCcyauOFPQQwuP/4LLXKzGx+c4gks9ETWbtiMd0dJV50iJjZAuEQSWr1oCQoTXNOBLIAuvzCpbz4+jHfFt7MFgSHSFKtN+iY6tcatvDuC3s4eqrGbk+wm9kC4BBJqvWYlRDZeNFSAP76p2/MuC4zs/nOIZJkPZHpD2WNWraog5VLOhl49eAstMrMbH5ziCS1xuwMZ0H2JVXP7DnseREze9sr9Kkpaauk3ZIGJd3a4vkuSQ+l53dIWp977ra0f7ekayarU9KfS3pZ0s70c+XMTrGYkdrsDGdBFiIHjg0zdOitWanPzGy+mvRTU1IZuBv4NLAJuF7SpqZiNwKHIuJy4C7gznTsJqAPuALYCnxFUrlAnf8yIq5MPztndIYFZT2RmQ9nwemvy31mz6FZqc/MbL4q8k/vLcBgRLwUESPAdmBbU5ltwP1p+xHgaklK+7dHxHBEvAwMpvqK1HlWVesNKrPUE7nogm4Wd5Z55lWHiJm9vRX51LwUeC33eCjta1kmImrAEWDVBMdOVucdkp6VdJekrgJtnLHZHM4ql8QH1i7jmT2HZ6U+M7P5qsinZqsxnuYZ43Zlprof4DbgvcBHgJXAF1s2SrpJ0oCkgQMHDrQqMiWzOZwF8OF3rOCFfUd5a6Q+a3Wamc03RUJkCFiXe7wW2NuujKQKsAw4OMGxbeuMiH2RGQb+jGzo6wwRcU9EbI6Izb29vQVOY2KzdbHhqA9dtoJaI3h2yL0RM3v7KvKp+RSwUdIGSZ1kE+X9TWX6gRvS9rXAE5Gtb+0H+tLqrQ3ARuDJieqUtCb9FvA54LmZnGBR1XpQmcF9s5p98LIVAB7SMrO3tcpkBSKiJukW4HGgDNwXEbsk3Q4MREQ/cC/wgKRBsh5IXzp2l6SHgeeBGnBzRNQBWtWZXvIbknrJhrx2Av9s9k63vWq9QU/XpH+OwlYu6WTD6iVeoWVmb2t6O1wQt3nz5hgYGJhRHf/oP/wNq3s6+dSmi2epVfCtZ4Z47mdH+O1ffs9YQP3aRy+btfrNzGZC0tMRsXkmdfiK9WS250QA/ueNq6nWG3z3+ddntV4zs/nCIZLMRYhcuLSbj71zFU+9cpB9R3z1upm9/ThEkuwuvrM3sT7qqvdeSHdHmcd+ss/30jKztx2HSFKbxSvW8xZ3Vvil913I3x84wY9e80otM3t7cYgkI7P0fSKtbNmwig2rl/Dozp+x++f+sioze/twiCS1RoPOORjOguw2KH0fWUd3pcw//09Pc+xUdU5ex8zsbHOIJNXa3AxnjVra3UHflst49eBJbn7wR74dipm9LThEkmpj7oazRm1YvYR/86vv569fPMA/uW8HR95yj8TMzm8OESAiZu3rcSdTb8B1m9fxzKuH2frHP+Dr/+OVOX9NM7O54hAB6o0ggjnviYz6wNrlXPeRdew7coqnfVsUMzuPOUSAWiO7fqNyFnoio6645ALWrVjED356gGq9cdZe18xsNjlEgJH0Id55lnoiAJL45Hsu5NDJKv07m++sb2Z2fpi929aex2r11BOZxVvBF/Hei5dy8QXd3P1Xg3xi42qe+Lv9vPLmCboqZVYu7uC6j1zGos7yWW2TmdlUOERgbDipo3J2O2ZZb6SX7U+9xkf/zfeArDc02jP66xff4E++8OE5XXpsZjYTDhFyIVIqjc2PnC3vv3QZHz94ksUdZTZdcgEXX9ANwJOvHOTRnXv53f/8HF/+X3+B7Du6zMzmF4cI2c0XAToqojZydkOkJPGPPnDJGfs/umEVl61czH94YpCX3zjBuy/u4QOXLufzm9c6UMxs3vA4CdnNFwEqpfn157j4gm6ufu+F/OzwW3zr6Z/xO996lr57fui7AZvZvOGeCPD8vqMA9HRVOHaqdo5bc5okrn7fRVz9vouICB7f9XN+8OIbfOnRXdz4iQ28eWIECd6xcjErl3S6h2JmZ12hEJG0Ffj3ZN+H/qcR8eWm57uArwMfBt4ErouIV9JztwE3AnXgX0TE4xPVKWkDsB1YCTwDfCEiRmZ2mu29fvQU/7p/Fx9Yu4xPbFzNXwwMzdVLzYgkrrniYgJ44Iev8sAPXx33/AXdFT7+rlV88j0XsmZZNydH6tQawaolnazu6WJJV5nujjLLF3V4ot7MZs2kISKpDNwNfAoYAp6S1B8Rz+eK3QgciojLJfUBdwLXSdoE9AFXAJcA35X07nRMuzrvBO6KiO2Svpbq/upsnGyzRiP4v/7ix5yq1rnruivP2hXr0yWJrVdczDtWLuZUrUFPV4VGI3jzxAivHz3FD186yOO7Jv4q3mWLOth25SV85hfW0IjgzeMj9HRVeGfvEi5e1k0ENCJY1FF2z8bMJlWkJ7IFGIyIlwAkbQxRZQQAAAhBSURBVAe2AfkQ2Qb8Xtp+BPh/lH0CbQO2R8Qw8LKkwVQfreqU9AJwFfBrqcz9qd45CZGv/+0r/PWLb/AHn3s/7+rtmYuXmHWS2HTJspbPRQT7jw0zXK3TWSkjwYnhGseHa4zUGlQbwZ43T/Dgjj18/W9fbVnHqFVLOnnvmqW8Y9USli/qYFFHmZ8fPcXew2/RCOjprnBBd4WergpLuzvYsHoJ7714Kb1LuxiuNTh2qsrg/uMM7j9OpVxizbJuLuju4OCJEY68VaWnq8Kqnk4WdZSpNYJ6I7t/Wb0R1BpBrZF9XfHaFYtZs6ybY6dq7D96inoEKxZ3smxRB40IqvWsbLUWlEtiVU/23PHhGgdPjPDm8REOnhjhxHCN5Ys7WL20i96eLlb3dNHdUaJaD07V6pyq1hmuNqiUxbJ0vs0hGhGcqjY4NlylJNFZKdFZLtFVKSEp3YMta0c5d83R6LlV6w1q9aDayH6PbneWS3R3lFncmfUWy7NwvVJE0Ej/IBg1Wquk3PbpfbMpIsaWqneUSpTO8jVYb0cR2T8YX3z9OKeqdd7V28PaFYvO+d+2SIhcCryWezwEfLRdmYioSToCrEr7f9h07KVpu1Wdq4DDEVFrUX7WdXeU+Ye/sIZ//NHL5uolzipJXJSWCLfz8Xeu4lS1zstvnKCrUmJxV4Xhap03jg9z9FSNkgTpzfrKGyfZuecwb1XrNAIWd5ZZvriDksSpaoPhap1TtfrY6rbzTUnQbkV3uSTKEigrJ5SFVZtzLZdEPVfZ6PHVRoOproPoLJfIf6aP20Yt9zciaDSy3/WIKb9m3liwcDpc8qEz1obxv8aeC4Lh2vjzlrIwKZeE82R66ukfMXndHSW+9usf5pPvufActapYiLT6T978Fm1Xpt3+VuNGE5U/s1HSTcBN6eFxSbtblSviK78+7uFq4I3p1nUOnG/tBbf5bHGbz45z2ub/5Q+mddhom98x09cvEiJDwLrc47VA882eRssMSaoAy4CDkxzbav8bwHJJldQbafVaAETEPcA9Bdo/JZIGImLzbNc7V8639oLbfLa4zWfHQm9zkZnkp4CNkjZI6iSbKO9vKtMP3JC2rwWeiOxihn6gT1JXWnW1EXiyXZ3pmO+nOkh1Pjr90zMzs7k0aU8kzXHcAjxOthz3vojYJel2YCAi+oF7gQfSxPlBslAglXuYbBK+BtwcEXWAVnWml/wisF3SHwA/SnWbmdk8JF/9PJ6km9JQ2XnhfGsvuM1ni9t8diz0NjtEzMxs2ub31XVmZjavOUQSSVsl7ZY0KOnWc9yW+yTtl/Rcbt9KSd+R9GL6vSLtl6T/O7X7WUkfyh1zQyr/oqQbWr3WLLV3naTvS3pB0i5Jv3EetLlb0pOSfpza/Ptp/wZJO9LrP5QWfpAWhzyU2rxD0vpcXbel/bslXTNXbc69XlnSjyR9+3xos6RXJP1E0k5JA2nfvH1vpNdaLukRSX+X3tcfn89tlvSe9Pcd/Tkq6TfPSpsjYsH/kE3u/z3wTqAT+DGw6Ry25x8AHwKey+37Q+DWtH0rcGfa/gzw38iusfkYsCPtXwm8lH6vSNsr5qi9a4APpe2lwE+BTfO8zQJ60nYHsCO15WGgL+3/GvDP0/b/CXwtbfcBD6XtTen90gVsSO+j8hy/P34LeBD4dno8r9sMvAKsbto3b98b6fXuB/6PtN0JLJ/vbc61vQz8nOwakDlv85yezPnyA3wceDz3+DbgtnPcpvWMD5HdwJq0vQbYnbb/BLi+uRxwPfAnuf3jys1x2x8luy/aedFmYDHZzT4/SnatUqX5fUG2kvDjabuSyqn5vZIvN0dtXQt8j+z2QN9ObZjvbX6FM0Nk3r43gAuAl0lzxudDm5va+cvA/3e22uzhrEyrW7vM2e1WpumiiNgHkH6P3uegXdvPyTmlIZMPkv3Lfl63OQ0L7QT2A98h+xd5u9vujLu1D5C/tc/Z/Dv/MfA7wOj9Lya6VdB8aXMA/6+kp5XdaQLm93vjncAB4M/SsOGfSloyz9uc1wd8M23PeZsdIpnCt1uZh6Z6y5m5a4jUA3wL+M2IODpR0Rb7znqbI6IeEVeS/et+C/C+CV7/nLdZ0q8A+yPi6fzuCV7/nLc5+cWI+BDwaeBmSf9ggrLzoc0VsuHkr0bEB4ETZENB7cyHNmcNyebDPgv8xWRFW+ybVpsdIpkit3Y5116XtAYg/d6f9rdr+1k9J0kdZAHyjYj4y/OhzaMi4jDwV2Rjw8uV3bqn+fXH2qbit/aZbb8IfFbSK2TfuXMVWc9kPreZiNibfu8H/jNZYM/n98YQMBQRO9LjR8hCZT63edSngWciYvQ7Iea8zQ6RTJFbu5xr+VvL5G8H0w/8k7Ta4mPAkdRtfRz4ZUkr0oqMX077Zp0kkd1Z4IWI+KPzpM29kpan7UXALwEv0P62O1O9tc+si4jbImJtRKwne48+ERH/eD63WdISSUtHt8n+mz7HPH5vRMTPgdckvSftuprsrhvzts0513N6KGu0bXPb5rme5DlffshWK/yUbFz8d89xW74J7AOqZP8yuJFsLPt7wIvp98pUVmRf8PX3wE+Azbl6/ndgMP38b3PY3k+QdXmfBXamn8/M8zZ/gOy2Os+Sfah9Ke1/J9kH6iDZkEBX2t+dHg+m59+Zq+t307nsBj59lt4jn+T06qx52+bUth+nn12j/2/N5/dGeq0rgYH0/vgvZCuV5nubF5N9s+yy3L45b7OvWDczs2nzcJaZmU2bQ8TMzKbNIWJmZtPmEDEzs2lziJiZ2bQ5RMzMbNocImZmNm0OETMzm7b/HwOtFPfhHjGyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "s2 = np.random.lognormal(5, 1, 10000)\n",
    "sns.distplot(s2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a20ec7278>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD4CAYAAADSIzzWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAQIElEQVR4nO3de2xUZ3rH8d+DxyHJspsQu0XUrDQgR93mj2o3Qe2irVZJSrY2VBWR9o9sKgX1IqQWEZr8USXBFJyQKG1ItcSK6qBeQqRett3dtkkwprAQhYuUrekab7a5MN1OsrBhgXEW4k0Aj/32jzkzHY9njMfmnGfMfj+S5XPe8877PAdOfh6fOSgWQhAAIHnzvBsAgJ9XBDAAOCGAAcAJAQwATghgAHCSqmdya2trSKfTMbUCANee1tZW7d27d28IoaPyWF0BnE6nNTAwcPU6A4CfA2bWWm2cWxAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnBDAAOCEAAYAJwQwADip6/8JdzX09PQok8mU9k+dOiVJamtrmzS3vb1dGzZsSKw3AEhS4gGcyWQ0+OZbGrvxFklS08fnJUmnL01spenj4aRbA4BEJR7AkjR24y365HOrJEk3vN0nSaX9ouI4AFyruAcMAE4IYABwQgADgBMCGACcEMAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnBDAAOCEAAYAJwQwADghgAHACQEMAE4IYABwQgADgBMCGACcEMAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOEgngnp4e9fT0JFHKpR4AzEQqiSKZTCaJMm71AGAmuAUBAE4IYABwQgADgBMCGACcEMAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnBDAAOCEAAYAJwQwADghgAHACQEMAE4IYABwQgADgBMCGACcEMAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcp7wbicPz4cUnSnXfe6dvINDQ3N6ulpUWnT5/W4sWLlcvldPnyZUnSvHnzlEqlNDo6qhCCnn32Wd1xxx3K5XLq7u7Wvffeq8cff1ypVEpmptHRUW3fvl033XST1q9fr0uXLmn+/Pl6/vnnJUnr16/X5cuXtW7dOr3wwguaP3++Fi1apLNnz6qnp0ft7e3KZDKledu3b1c6nVZ3d7fWrFmjJ554QkuXLtUzzzyjlpYW5XI5bd68Wfl8XhcvXtT777+vtrY2DQ8Pa8mSJXr66afV0tJSOtdcLqeuri6NjY2pqalJ27Ztm3C8XHHtEMKkecXz37JlS9XxBx98UM8995y2bNkiSdq8ebNGR0dlZlPWrbVuLZXzy/cl1bXWbNXbexI1PXqKQ5zn0bR169ZpT965c+fWdevW1V2kv79fktTZ2an+/n598OHPlG+9VZLUfO6EJJX2i5rPndAvLVygzs7Ouuu9+OKLdb/Gy/j4uEZGRiRJIyMjGhsbKx0LIUzYP3r0qO6//3719vbq0KFDOnz4sMbHxzU+Pl6ad/ToUQ0MDCiXy0mSxsbGNDQ0pNdff700duzYsdKx8+fPK5/Pa2hoSGvWrNHDDz9cmnf06FFduHBBhw4d0pEjRzQ+Pq4PP/xQly5d0ooVK9Tb26vDhw8rl8vp/PnzkqSPPvpI+Xxew8PDpXlFvb29OnLkiHK5nM6dOzfpeLni2tXmFc//4sWLVceHhob07rvv6uLFixocHNThw4c1PDx8xbq11q2lcn75/uDgYF1rzVa9vSdR06OnOFyN8+ju7v5g69atOyvHr7lbEHPhXe9MjYyM6MCBA+rv71cIQfl8vuqcbDY7YSybzU4aq5TNZnXw4MEJ80ZGRrR79+5JtXbv3q1MJlP6wVpLX19fKcxzuZz27Nkz4fiePXtKx8vlcrkJa5fPKx4LIai/v7/qeDabVQhBe/bsmVSzVt1a69ZSOb/451Fed7przVa9vSdR06OnOMR9Honcgjh16pQ++eQTbdy4UZlMRvMuhyu+Zt7FC8pkPtLGjRsT6HDueOqpp2Jb+8knn5w0Vv4OvCifz2vbtm0aHR2dcr3R0VG99NJLeuihh7Rr165JPzDKj5fbtWvXhLUr1xkfHy/1Vm28/HUhTL7WqtWttW4tlfO3bdtW2i/vfTprzVa9vSdR06OnOMR9Hld8B2xm68xswMwGzp49e9UKY2by+XzVd75Xa+3pKr7LvJJ9+/ZJkvbv3z9pfgihdLxc5dzyefv37y/1mc/nq46Xv66aanVrrVtL5fxsNlvaDyGUak9nrdmqt/ckanr0FIe4z+OK74BDCDsl7ZSk5cuXX/m/uCra2tokSTt27NDGjRt17Ic/ueJrxq//jNqXLdKOHTvqqnUt34KQpFSq8FcWRwinUqlpr5tOp/Xee+9dMYTvueceSdLKlSv1yiuvTJhvZqXj5Srnls9buXKl+vr6lM/nlUqlqo6Xr1+tv2p1a61bS+X8JUuW6OTJk8rn8zIzSYUgns5as1Vv70nU9OgpDnGfxzV3D/ha99hjj2nevHj+2jZt2jRprKmpadJYKpVSV1eXmpubp1yvublZDzzwgCRp7dq1pR8e1Y6XW7t27YS1K9cpnn9TU1PV8fLXVeuxWt1a69ZSOb+rq6u039zcXDrX6aw1W/X2nkRNj57iEPd5XHMB/Nprr3m3EJsFCxbo7rvvVkdHh8xsUqAV56TT6Qlj6XR60lildDqtu+66a8K8BQsWaPXq1ZNqrV69Wu3t7ero6JhyzVWrVpUe22lpaZn0REtnZ2fVx3paWlomrF0+r3jMzNTR0VF1PJ1Oy8zU2dlZ9SmaanVrrVtL5fzin0d53emuNVv19p5ETY+e4hD3eVyTzwHPJfU8B9zd3S2p8FM5m81WfQ64u7t70nPAXV1dkqZ+Drg4p6urqzSvu7tb6XRa2Wx2wnPA5e86M5lMzeeAq70rOnHiROk54KneTRTXDiFUXSebzdYcLz4HXDyeyWQmPAdcq26tdafqsXx+5X49a81Wvb0nUdOjpzjEeR42nQ9SipYvXx4GBgbqLlJ8kqH8HvAnn1slSbrh7T5JKu0X3fB2n+6YwT3gynoA4M3MjoUQlleOX3O3IABgriCAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnBDAAOCEAAYAJwQwADghgAHACQEMAE4IYABwQgADgBMCGACcEMAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnKSSKNLe3p5EGbd6ADATiQTwhg0bkijjVg8AZoJbEADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnBDAAOCEAAYAJwQwADghgAHACQEMAE4IYABwQgADgBMCGACcEMAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnBDAAOCEAAYAJwQwADghgAHACQEMAE4IYABwQgADgBMCGACcpDyKNn08rBve7ou2c5JU2i+fIy1KujUASEziAdze3j5h/9SpvCSpra0ybBdNmgsA15LEA3jDhg1JlwSAhsQ9YABwQgADgBMCGACcEMAA4IQABgAnBDAAOCGAAcAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACcEMAA4IYABwAkBDABOCGAAcEIAA4ATAhgAnBDAAODEQgjTn2x2VtJ7M6zVKuncDF/rib6TRd/Jou/4nZOkEEJH5YG6Ang2zGwghLA8kWJXEX0ni76TRd++uAUBAE4IYABwkmQA70yw1tVE38mi72TRt6PE7gEDACbiFgQAOCGAAcBJ7AFsZh1m9o6ZZczskbjrTaOfvzWzM2b2ZtnYLWa2z8xORN8XRuNmZs9FvQ+Z2e1lr1kbzT9hZmsT6PuzZnbQzN4ysx+Y2ca50LuZXW9m3zWz41Hf3dH4UjN7I+rhG2Z2XTQ+P9rPRMfTZWs9Go2/Y2a/FWffZTWbzOx7ZvbqXOnbzLJm9n0zGzSzgWisoa+TqN7NZvZNM3s7us5XzIW+ZyWEENuXpCZJ/yNpmaTrJB2XdFucNafR05cl3S7pzbKxv5D0SLT9iKQ/j7ZXSdojySR9UdIb0fgtkn4YfV8YbS+Mue/Fkm6Ptj8t6V1JtzV671H9BdF2s6Q3on7+WdJ90XivpD+Ktv9YUm+0fZ+kb0Tbt0XXz3xJS6PrqimB6+VhSf8g6dVov+H7lpSV1Fox1tDXSVRzl6Q/jLavk3TzXOh7Vucc8x/oCkl7y/YflfSo+0lLaU0M4HckLY62F0t6J9p+QdLXKudJ+pqkF8rGJ8xL6Bz+XdI9c6l3STdK+i9Jv67Cvw5KVV4nkvZKWhFtp6J5VnntlM+Lsd8lkr4j6W5Jr0Z9zIW+s5ocwA19nUj6jKT/VfRgwFzpe7Zfcd+CaJP0o7L9k9FYo1kUQvhAkqLvvxiN1+rf9byiX2+/oMK7yYbvPfo1flDSGUn7VHgX+NMQQr5KD6X+ouPnJbV49C3p65L+VNJ4tN+iudF3kPQfZnbMzNZFY41+nSyTdFbS30W3fP7azD41B/qelbgD2KqMzaXn3mr173ZeZrZA0rck/UkI4cJUU6uMufQeQhgLIXxehXeUvybpV6booSH6NrPflnQmhHCsfHiKHhqi78iXQgi3S+qUtN7MvjzF3EbpO6XCrcG/CiF8QdLPVLjlUEuj9D0rcQfwSUmfLdtfIunHMdeciZ+Y2WJJir6ficZr9e9yXmbWrEL4/n0I4dvR8JzoXZJCCD+V9JoK9+xuNrNUlR5K/UXHb5I0rOT7/pKk3zGzrKR/UuE2xNfnQN8KIfw4+n5G0r+q8EOv0a+Tk5JOhhDeiPa/qUIgN3rfsxJ3AP+npFujT46vU+HDiZdjrjkTL0sqflq6VoX7q8XxB6JPXL8o6Xz0a9BeSV8xs4XRp7JficZiY2Ym6W8kvRVC+Mu50ruZ/YKZ3Rxt3yBppaS3JB2U9NUafRfP56uSDoTCzbyXJd0XPW2wVNKtkr4bV98hhEdDCEtCCGkVrtsDIYTfbfS+zexTZvbp4rYKf79vqsGvkxDCaUk/MrNfjoZ+U9J/N3rfsxb3TWYVPq18V4X7fpu8b3pL+kdJH0gaVeGn5R+ocK/uO5JORN9vieaapOej3r8vaXnZOr8vKRN9/V4Cff+GCr9KDUkajL5WNXrvkn5V0veivt+U9GfR+DIVgigj6V8kzY/Gr4/2M9HxZWVrbYrO5x1JnQleM3fq/5+CaOi+o/6OR18/KP431+jXSVTv85IGomvl31R4iqHh+57NF/8UGQCc8C/hAMAJAQwATghgAHBCAAOAEwIYAJwQwADghAAGACf/BzNhDQA3Bl9GAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.boxplot(s2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a210c8470>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de3jU53nn//c9I2l0Po8ASQgJIcDYYGPLEION7dpJcJKfSdp0g91snfzcsk7tJrtpr67T5kp23U03TX6/NLlaN4mbOE0TE+I4Tkqy2MRHsI3BEuczOoBOgM7ofJqZe//QQGUh0AhG+o5m7td16WLme5j5SNbcfvR8n+/ziKpijDEmermcDmCMMWZ6WaE3xpgoZ4XeGGOinBV6Y4yJclbojTEmysU5HWC83NxcLS4udjqGMcbMKnv37m1TVe9E+yKu0BcXF1NZWel0DGOMmVVEpO5K+6zrxhhjopwVemOMiXJW6I0xJspZoTfGmChnhd4YY6KcFXpjjIlyIRV6EVkvIidFpFpEnrzKcZ8UERWR8jHbvhQ876SIfDgcoY0xxoRu0nH0IuIGngY+CDQCFSKyVVWPjTsuDfg8sGfMtmXARuBGIB94VUQWq6o/fN+CMcaYqwmlRb8KqFbVWlUdBrYAGyY47m+BbwCDY7ZtALao6pCqngaqg69njDFmhoRyZ2wB0DDmeSOweuwBIrISmK+qvxWRvxx37u5x5xaMfwMR2QRsAigqKgotuTFXsXlP/YTbH14d+u/XRK8xlfONiRShtOhlgm2XlqUSERfwD8BfTPXcSxtUn1HVclUt93onnKrBGGPMNQqlRd8IzB/zvBA4O+Z5GnAT8KaIAMwFtorIgyGca4zjrtT6NyZahNKirwDKRKRERBIYvbi69eJOVe1S1VxVLVbVYka7ah5U1crgcRtFxCMiJUAZ8F7YvwtjjDFXNGmLXlV9IvIEsB1wA8+q6lEReQqoVNWtVzn3qIg8DxwDfMDjNuLGGGNmVkjTFKvqNmDbuG1fucKx94x7/jXga9eYz5gZ09E3zOm2Puo7+hjyBUhwu0hPimdFYQZ5aYlOxzPmmkXcfPTGzKQhn58DDZ3sru2gvqMfgKR4N8kJbkb8AXqHfLx+ooX5WUl8cNlch9Mac22s0JuY5A8ov9zXyLdfOcXZrkFyUhJ44Ka5lM1JIy/Ng2t0YAE9gyMcbLjA7tMdPPvOaUb8AZ58YCkpHvvomNnDfltNzDnc2MVf/uIgJ5t7uLkwgw/dOJdFeamXivtYaYnx3FnmZfXCHF451sxP99TxTk0b//qZVRTlJDuQ3pips0nNTMwIqPL0G9V84p/foWtghO/+0a38+vG1LJ6TNmGRHyve7eIjy+fx3J+spr13mN//7jscaLgwQ8mNuT5W6E1M8AeULRUNfHP7ST5801y2/9d1PLB8HjJJgR9vTWkuv/zcGpIS3Gx85l3ermqbpsTGhI8VehP1/AHl+coGjjR18dcfWco/PbSSjOT4a369RXmpvPi5tRTnpPDojyvYVW3F3kQ2K/QmqqkqL+5r5HBTFw/cNJdN60qn3IqfiDfNw3N/sprinBT+3x9XsLu2PQxpjZkedjHWRLW9dZ3sb7jAfUvzuKssvPMo5aR6eO5PV/PAd97isz+q4HP3lJKb6rm03yZAM5HCWvQmarX1DPGbQ2dZ6E3h3qV50/IeuakeHrmjGBH4t3frGBi2G79N5LEWvYlK/oDy88oG4lwu/vC2+ZOOqgnVRBOgZack8PDqIp59+zRbKup5ZE1x2N7PmHCwFr2JSrtq2mi6MMDHVxaQkXTtF15DtTA3lY+tyKeqpZfKM53T/n7GTIUVehN1OvqGeeNkC4vnpLK8IGPG3nd1STYLvSm8dOQcXQMjM/a+xkzGum5M1PnOq6cY9gV44KZ5M/q+IsInbingO69VsfXgWdIT4y4b4WMXaI0TrNCbqFLd0sNP99Rze3E2c9Ivn3FyuhcZyUn1cP8Nc3j56HlOnO/hhnnp0/p+xoTCum5MVPn7l0+SHO/mvhvmOJZh7aJcslMSeO1EM6qXrZxpzIyzQm+ixuHGLl451syfrltIqoOzS7pdwt2LvZy9MEhVS69jOYy5yAq9iRr/8OopMpLi+ezaYqejsLIok4ykeN482eJ0FGNCK/Qisl5ETopItYg8OcH+x0TksIgcEJG3RWRZcHuxiAwEtx8Qke+F+xswBuBAwwVeP9HCpnULSUuc/uGUk4lzubirLJcz7f2cbutzOo6JcZMWehFxA08DDwDLgIcuFvIxNqvqclW9BfgG8K0x+2pU9Zbg12PhCm7MWN9+9RRZyfE8sqbY6SiXlC/IJiXBzc5TrU5HMTEulI7MVUC1qtYCiMgWYAOjC34DoKrdY45PAewKlJkxf//SCd482cqHl81h64GzTse5JCHOxe0l2ew42Ur3wAjpM3DjljETCaXrpgBoGPO8MbjtfUTkcRGpYbRF//kxu0pEZL+I7BCRuyZ6AxHZJCKVIlLZ2mqtHzM1b5xsISnezQcW5jgd5TK3FmWhYIuUGEeFUugnmrTjsha7qj6tqqXAfwe+HNx8DihS1ZXAF4HNInLZwGJVfUZVy1W13OsN7wyDJrodPdvFifM9rFmUgyfe7XScy+SmeijKTmZffacNtTSOCaXQNwLzxzwvBK729/EW4OMAqjqkqu3Bx3uBGmDxtUU15nJPv1GNJ87FmoW5Tke5oluLsmjpGaLpwoDTUUyMCqXQVwBlIlIiIgnARmDr2ANEpGzM048CVcHt3uDFXERkIVAG1IYjuDFVzT28dOQ8dyzMISkh8lrzFy0vyCDOJeyrt+4b44xJL8aqqk9EngC2A27gWVU9KiJPAZWquhV4QkTuB0aATuCR4OnrgKdExAf4gcdUtWM6vhETe55+o5rEODdrF0Vuax4gKcHNsvx0DjZcYMjnxxMXuf9TMtEppNsHVXUbsG3ctq+MefyFK5z3S+CX1xPQmImcaetj68GzPHpnCSkO3gUbqpXzMznU2MWu6vZpWwTFmCuxO2PNrPTPb1YT53bxp+sWOh0lJKXeVDxxLn53rNnpKCYGWaE3s05jZz8v7mviodvnk5d2+QyVkSjO7aJsThqvHm8mELDRN2ZmWaE3s873dtQgAv/l7lKno0zJsnlptPYMcbDRLsqamWWF3swqbb1DPF/ZyB/cWkh+ZpLTcaZkyZx03C6x7hsz46zQm1nlr144xLAvwNyMRDbvqZ/2hUTCKSnBzeqSbF6xQm9mmBV6M2sMjvjZXdvO0rlps6ZvfrwPLZtDdUuvzWhpZpQVejNr/Gp/E/3Dfu6M8HHzV3P/stGVr145dt7hJCaWWKE3s0IgoPzgrVryMxIpyU1xOs41K8xKZuncNN48aZP3mZljhd7MCjurWqlp7ePOslxEJppnb/ZYt9hLxZkO+oZ8TkcxMSLybyk0MWn8Rdaf7q4jJcHNTQUZDiUKn3VlXp7ZWcvu2nZHFzE3scNa9Cbi9QyOcOJ8N7cWZRHnmt2/spv31FPT2ku8W/iXt07PqlFDZvaa3Z8aExP21XUSULi9ONvpKGER73ZRkptCVXOP01FMjLBCbyJaQJWKuk5KclPITfM4HSdsyvLSaO8bpqNv2OkoJgZYoTcRrba1j46+YW4vznI6SliVzUkFoKrFWvVm+lmhNxGtsq6DpHg3N+bP/ouwY3lTPWQmxVPV3Ot0FBMDrNCbiDXsC3D8XDfLCzKId0fXr6qIUDYnlZrWXnz+gNNxTJSLrk+PiSrHz3Uz4ldWzI+u1vxFpd5UhnwBDjV1OR3FRLmQCr2IrBeRkyJSLSJPTrD/MRE5LCIHRORtEVk2Zt+XguedFJEPhzO8iW4HGy+QnhhHcc7svRP2ahZ6R/vpd1W3OZzERLtJC31wce+ngQeAZcBDYwt50GZVXa6qtwDfAL4VPHcZo4uJ3wisB/754mLhxlxN/7CPquZeVhRm4prld8JeSaonjnkZibxT3e50FBPlQmnRrwKqVbVWVYeBLcCGsQeoaveYpynAxSV0NgBbVHVIVU8D1cHXM+aqjp7txq/KzYWZTkeZVqXeVPbWdzI44nc6iolioRT6AqBhzPPG4Lb3EZHHRaSG0Rb956d47iYRqRSRytZWm+zJjHbb5KQkkJ85O6cjDtVCbwrDvgB76zqdjmKiWCiFfqK/my9b9FJVn1bVUuC/A1+e4rnPqGq5qpZ7vd4QIplo1tozxOnWPm6enznrJzCbTElOCnEu4R3rpzfTKJRC3wjMH/O8EDh7leO3AB+/xnON4dXjzShwY36601GmnSfezc3zM3mnxvrpzfQJpdBXAGUiUiIiCYxeXN069gARKRvz9KNAVfDxVmCjiHhEpAQoA967/tgmmv3u6HmykuOZmx7d3TYXrS3N4XDjBboHR5yOYqLUpIVeVX3AE8B24DjwvKoeFZGnROTB4GFPiMhRETkAfBF4JHjuUeB54BjwMvC4qtpVJ3NFPYMjvFPdzo35GVHfbXPRHaW5BBT21HY4HcVEqZDmo1fVbcC2cdu+MubxF65y7teAr11rQBNbdpxqZdgf4IZ50d9tc9GtCzJJjHfxTnUbH1xm89Ob8LM7Y01E+d3RZnJSEliQk+x0lBnjiXNze3E2u2rsgqyZHlboTcQY9gV440QL998wJ2pvkrqSO0pzONXcS2vPkNNRTBSyQm8ixru17fQM+fjwTbHXfbG2NBfAWvVmWtiasSZivHqsmaR4N2tKc3lxX5PTcWbM5j31BFRJjHfxk3fr6BsaHa/w8Ooih5OZaGEtehMRVJXXT7SwdlEuifGxNx2SS4SFuaPTFhsTblboTUQ41dxL04UB7rshz+kojlnoTaGzf8SWFzRhZ4XeRITXT7QAcO+S2C30pcFpi61Vb8LNCr2JCG+caGHZvHTmZsTG3bATyUvzkJYYZ4XehJ1djDWO+8FbtVSc6eDuJV4276l3Oo5jRIRSbypVLb2oXjb3nzHXzFr0xnFVzb0osHROmtNRHLcwN4W+IR/NNp7ehJEVeuO4k809JCe4KcyOnbthr6Q0L9hP32LdNyZ8rNAbR/kDyqnmHhbPSYu5u2EnkpWcQHZKgvXTm7CyQm8cdaSpi/5hP2XBlqyBUm8Kp9v68PkDTkcxUcIKvXHUzlOjS0eWWf/8JaXeVIZ8AQ43dTkdxUQJK/TGUTurWsnPTCTVYwPALloYHE+/y1adMmFihd44pntwhH31FyjLs9b8WKmeOOamJ9o6siZsrNAbx+yqbscfUMrmWP/8eKXeFCrrOhkcsQXZzPULqdCLyHoROSki1SLy5AT7vygix0TkkIi8JiILxuzzi8iB4NfW8eea2LWzqpWUBDdFNqzyMqXeVIZ9AfbVdTodxUSBSTtGRcQNPA18EGgEKkRkq6oeG3PYfqBcVftF5HPAN4BPBfcNqOotYc5tZqmLd76qKi8dPkdRdjJxLvvDcrzi3BTcLmFXTTtrFuU6HcfMcqFcAVsFVKtqLYCIbAE2MLrgNwCq+saY43cDnw5nSBN92nuH6ewf4a4yr9NRIlJivJv8jET+/UAT+ZlJl7bbHPXmWoTSlCoAGsY8bwxuu5JHgZfGPE8UkUoR2S0iH5/oBBHZFDymsrW1NYRIZrY71dIDYOPnr6I0L5WmCwPWT2+uWyiFfqLbFSeccUlEPg2UA98cs7lIVcuBh4Fvi0jpZS+m+oyqlqtquddrLbxYUNXcS3ZKAjmpHqejRKxSbyoBhTNtfU5HMbNcKIW+EZg/5nkhcHb8QSJyP/A3wIOqemlGJlU9G/y3FngTWHkdeU0U8PkD1Lb1Wmt+EqPXL8SmQzDXLZRCXwGUiUiJiCQAG4H3jZ4RkZXA9xkt8i1jtmeJiCf4OBdYy5i+fROb6jr6GfEri+1u2KuKd7soykmm1lr05jpNWuhV1Qc8AWwHjgPPq+pREXlKRB4MHvZNIBX4xbhhlDcAlSJyEHgD+Pq40TomBlU19+KS0Sl5zdWVelM51zVI75DP6ShmFgvpvnNV3QZsG7ftK2Me33+F83YBy68noIk+VS09LMhJwRODi4BPVak3lVdo5nRbH8sLMpyOY2YpG8BsZlTP4Ajnugatfz5EBZlJeOJcNj+9uS5W6M2Mqg4WLJutMjRul1CSm2IXZM11sUJvZlRVSy8pCW7mxfAi4FO10JtKe98wF/qHnY5iZikr9GbGBAJKVXMPZbaa1JSUekcvWte22ugbc22s0JsZc+xcN322mtSUzUlPJDnBbd035ppZoTczZkdwNalFVuinxCVCqTeVmtZeVCe8Kd2Yq7JCb2bMjlOtzMtIJC0x3ukos85Cbwrdgz5O281T5hpYoTczomdwhH11nXY37DVaFFxe8B1bXtBcAyv0Zka8W9OOL6DWP3+NslMSyEiK590aW17QTJ0VejMjLq0mlWOrSV0LCfbT76ppJxCwfnozNVbozYzYeaqNO0pzbDWp67AoL4UL/SMcPdvtdBQzy9inzky7M2191Hf0s26xrTVwPUqD/fQ7q2xxHjM1VujNtLtYmNbZsoHXJS0xnqVz03i7yvrpzdRYoTfTbuepVoqykym2aYmv27rFXvbWddI/bNMWm9BZoTfTatgXYFdNO+sW5zodJSrcuSiXYX+APac7nI5iZhEr9GZaVdZ10D/st26bMFlVkk1CnMu6b8yUhFToRWS9iJwUkWoReXKC/V8UkWMickhEXhORBWP2PSIiVcGvR8IZ3kS+nafaiHMJd5TmOB0lKiTGu7m9OMsKvZmSSQu9iLiBp4EHgGXAQyKybNxh+4FyVV0BvAB8I3huNvBVYDWwCviqiGSFL76JdDtPtXLrgiyb9iCM7irzcrK5h+buQaejmFkilBb9KqBaVWtVdRjYAmwYe4CqvqGq/cGnu4HC4OMPA6+oaoeqdgKvAOvDE91EutaeIY6d6+ZuG1YZVncuGr3esfOUDbM0oQml0BcADWOeNwa3XcmjwEtTOVdENolIpYhUtrbaL2+0eCs4rNIKfXgtm5eON83Dm1boTYhCKfQTrRAx4T3YIvJpoBz45lTOVdVnVLVcVcu9XisK0eK1Ey3kpnpYNi/d6ShRxeUS7l3iZeepVnz+gNNxzCwQSqFvBOaPeV4InB1/kIjcD/wN8KCqDk3lXBN9RvwBdp5s5b6lebhctppUuN27JI+eQR976zqdjmJmgbgQjqkAykSkBGgCNgIPjz1ARFYC3wfWq2rLmF3bgb8bcwH2Q8CXrju1iXhff+kEPUM+4t3C5j31TseJOmvLcolzCW+cbGX1QhvRZK5u0ha9qvqAJxgt2seB51X1qIg8JSIPBg/7JpAK/EJEDojI1uC5HcDfMvo/iwrgqeA2E+VOnOsmziWU2rTE0yI9MZ7y4izePNky+cEm5oXSokdVtwHbxm37ypjH91/l3GeBZ681oJl9VJUT53tY6E3BE+d2Ok7U+r2lefzdthM0XRigIDPJ6TgmgtmdsSbsatv6aO8bZulcuwg7ne5dkgdgrXozKSv0JuxeO94MwJK5tmzgdFqUl0pBZhJvnLBCb64upK4bY6bi1eMtzE1PJCs5wekoUWf8he2i7GTeqmqjb8hHisc+zmZi1qI3YdXWO0TlmQ5umGet+ZlwY346Q74AO+zmKXMVVuhNWL16rJmAwo35GU5HiQkLclLITklg+9HzTkcxEcwKvQmrl4+epyg7mXkZiU5HiQlul3D/DXm8fryFYZ/dJWsmZoXehE334AjvVLex/qa5iNjdsDNl/U1z6RnysavGpi42E7NCb8LmjRMtjPiVD9841+koMWVNaS6pnjjrvjFXZIXehM1Lh8+Tl+Zh5fxMp6PElMR4N/cs8fLKsWb8gQnnGzQxzgq9uW6b99Tzr++c4bUTzZTkprClomHyk0xYrb9pLm29w7xna8maCdjAWxMWJ5t7GPErNxXYaJuZtnlPPcO+AAluF9965RSfaBtd8uHh1UUOJzORwlr0JiwONV4g1RNHSW6K01FiUkKci2X56Rxp6sIXsNE35v2s0JvrNjji5+T5HpYXZOCy0TaOWVGYwcCIn+rmXqejmAhjhd5ct+PnuvEFlJsLrdvGSYvyUkmKd3Ow8YLTUUyEsUJvrtvBxgtkJsczPzvZ6SgxLc7lYnlBBsfOddvNU+Z9rNCb69LRN0x1Sy8rCjLsJqkIsGJ+BiN+5fj5bqejmAhihd5cl5ePnCegsKLQxs5HguKcFDKS4tlfb2vJmv8QUqEXkfUiclJEqkXkyQn2rxORfSLiE5FPjtvnDy4veGmJQRM9th5sIjfVY3PbRAiXCCvnZ1LV3Etz96DTcUyEmLTQi4gbeBp4AFgGPCQiy8YdVg98Btg8wUsMqOotwa8HJ9hvZqnm7kH2nO7g5kLrtokkty7IQoEX9zU5HcVEiFBa9KuAalWtVdVhYAuwYewBqnpGVQ8BdgUohvz20DnUum0iTm6qhwU5yfxibwOqNiWCCa3QFwBj72lvDG4LVaKIVIrIbhH5+EQHiMim4DGVra22gMJs8ZuDZ7kxPx1vmsfpKGac24qyqG3tY3+DDbU0oRX6if4mn0ozoUhVy4GHgW+LSOllL6b6jKqWq2q51+udwksbp9S393Og4QL/z835TkcxE1hekEFSvJtfVDY6HcVEgFAKfSMwf8zzQuBsqG+gqmeD/9YCbwIrp5DPRKjfHBr9FfjYinkOJzET8cS7eWD5XH578Cz9wz6n4xiHhVLoK4AyESkRkQRgIxDS6BkRyRIRT/BxLrAWOHatYY3zNu+pZ/Oeen7ybh1F2cnsPGWLXUSqh1cV0TPkY+uBkNtlJkpNWuhV1Qc8AWwHjgPPq+pREXlKRB4EEJHbRaQR+EPg+yJyNHj6DUCliBwE3gC+rqpW6Ge5892DnO8etCkPItxtC7JYOjeN5/bUOx3FOCykaYpVdRuwbdy2r4x5XMFol87483YBy68zo4kwBxsu4BJYbqNtItrP3mtg8Zw0th48yzdePkFhVrJNXRyj7M5YMyUBVQ40XKAsL41Ujy1nEOlumZ9JgtvFHluQJKZZoTdTUtfeT9fACDfbcoGzQmK8m5vnZ3Co8QIDw36n4xiHWKE3U3KgoZMEt4tl89KdjmJCtLokhxG/stfmv4lZVuhNyIZ8fg43dbEsP52EOPvVmS3yM5NYkJPMuzVttnh4jLJPqwnZmydbGRwJcIt128w6a0tz6ewf4ZVj552OYhxghd6E7Nf7m0jxxFHqTXU6ipmiZfnpZCXH88O3TzsdxTjACr0JSdfACK+daGFFYQZul81UOdu4RLijNJeKM50csqUGY44VehOSl4+cY9gXYKV128xa5QuySPXEWas+BlmhNyH59f6zlOSmUJCZ5HQUc40S491svH0+vz10joaOfqfjmBlkhd5M6lzXALtPt7PhlnxbYGSWe/SuElwC//JWrdNRzAyyQm8mtfXAWVTh47dMZRkCE4nmZSTx+ysL+XlFA609Q07HMTPECr2Z1K8PnOWW+ZkU56Y4HcWEwaa7FzLsD/Cvu6yvPlZYoTdXdfJ8D8fPdfOJldaajxal3lTW3ziXf3u3jp7BEafjmBlghd5c1a8PNOF2CR+1BUaiyufuKaVn0GdTGMcIK/TmigIB5d/3N3FXWS65qbYubDRZUZjJnYty+eHbpxkcscnOop3NM2smtHlPPafb+jjbNcidZV42W8svKoz977h4ThpvV7fx4r4mm6c+ylmL3lyRzVQZ3Uq9KRRmJfH9nTX4/AGn45hpFFKhF5H1InJSRKpF5MkJ9q8TkX0i4hORT47b94iIVAW/HglXcDO9fP6AzVQZ5USEdWVe6tr7eemITXYWzSb9BIuIG3gaeABYBjwkIsvGHVYPfAbYPO7cbOCrwGpgFfBVEcm6/thmup0432MzVcaAZfnpLPSm8N03a1C1KYyjVShNtVVAtarWquowsAXYMPYAVT2jqoeA8X//fRh4RVU7VLUTeAVYH4bcZprtreskPTGORXk2U2U0c4nw2N2lHDvXzY5TrU7HMdMklIuxBUDDmOeNjLbQQzHRuZcNyBaRTcAmgKIiuyjktPNdg5xq7uHuJV5cNuVB1Bsc8ZOeGMf//M0xzl4YvLTdLtBGj1Ba9BN90kP9Gy+kc1X1GVUtV9Vyr9cb4kub6fLLfY0ocFuR9bLFgjiXizvLvJxu66O+vc/pOGYahFLoG4H5Y54XAmdDfP3rOdc4QFV5vrKBktwUcmzsfMy4vTiLpHi3dd9EqVAKfQVQJiIlIpIAbAS2hvj624EPiUhW8CLsh4LbTIR673QHde39lC+w1nws8cS5uaM0h+Pne2juHpz8BDOrTFroVdUHPMFogT4OPK+qR0XkKRF5EEBEbheRRuAPge+LyNHguR3A3zL6P4sK4KngNhOhtlQ0kOqJ48b8DKejmBm2ZmEO8W5hp7Xqo05Id8aq6jZg27htXxnzuILRbpmJzn0WePY6MpoZ0tozxG8PneWPVi+wsfMxKNkTx6ribN6tbef+ZXOcjmPCyD7N5pKfvVfPiF/5z3cscDqKccjaRbkIwttVbU5HMWFkhd4AMOIP8NyeOu4qy6XUa2PnY1VmcgK3zM+ksq6D9l5bmCRaWKE3AGw/ep7m7iE+s6bY6SjGYXctzsXnV370zhmno5gwsUJvAPjxrjMUZSdzz5I8p6MYh+WlJbIsP50fv3uGbluYJCpYoTfsreug4kwnf3zHAtwuuxPWwL1L8ugZ9PGTd+ucjmLCwAq94R9fryY7JcFueTeX5Gcmcc8SLz98+zQDw7YwyWxnhT7GHWq8wJsnW3n0zhKSE2wdGvMfnrh3ER19w/zsPVt0ZrazT3aM+8fXq0mMd5EU77ZVpMz7lBdn84GF2Xx3Rw0PrSoiKcHtdCRzjaxFH8OOn+vmlWPNrCnNJTHePsTmcn/xoSW09gzx43fPOB3FXAcr9DFKVfm7bcdJS4xjTWmO03FMhLq9OJt7lnj53o4aG4Ezi1mhj1Fvnmzlrao2vnBfmfXNm6v6yw8t4UL/CD9467TTUcw1skIfg0b8Af7X/zlGSW4Kf3xHsdNxTIS7qSCDB26ayw/fqqW1x+6WnY2s0Meg53bXUdPax19/5AabvMxc0eY99Q2i6mUAAAzTSURBVJe+ls5NZ3AkwDdePuF0LHMN7FMeY5q7B/n/XznF2kU53H+D3QVrQuNN87B2UQ6/2NvIvvpOp+OYKbJCH0NUlb/51RFG/AG+9vHliK0Ha6bg3iV55KV5+Oq/H8UfCHU1URMJrNDHkN8cOserx5u5d0keu2raL/1ZbkwoPPFu/uajN3C4qctuopplrNDHiLbeIf7H1qMUZiWxdlGu03HMLPXgzfmsXZTD/952nIaOfqfjmBCFVOhFZL2InBSRahF5coL9HhH5eXD/HhEpDm4vFpEBETkQ/PpeeOObUAQCyl88f5DeIR9/cGshLuuyMdfoZ+81sKY0F19AeeTZ9/jpbpv0bDaYdAC1iLiBp4EPAo1AhYhsVdVjYw57FOhU1UUishH4e+BTwX01qnpLmHObSYztknm7qpUdp1p58OZ85qQnOpjKRIOs5AQ+snwev9rfxJ7adj79AVuRLNKF0qJfBVSraq2qDgNbgA3jjtkA/Dj4+AXgPrErfRGhqXOA7UebWTYvndUl2U7HMVGifEEWi+ek8tKR85w43+10HDOJUAp9AdAw5nljcNuEx6iqD+gCLt5XXyIi+0Vkh4jcNdEbiMgmEakUkcrWVluBPlz6hnw8914dqYlx/P6tBTbKxoSNiPAHtxaSFO/mz57bR9+Qz+lI5ipCKfQTVYfxY6uudMw5oEhVVwJfBDaLSPplB6o+o6rlqlru9XpDiGQm4w8oP69ooHfQxx+tLrJpDkzYpSXG859un8+Ztj6+/OsjqNqQy0gVSqFvBOaPeV4InL3SMSISB2QAHao6pKrtAKq6F6gBFl9vaDO53x07T3VrLw/enE9hVrLTcUyUKvWm8oX7FvOr/U381IbqRqxQCn0FUCYiJSKSAGwEto47ZivwSPDxJ4HXVVVFxBu8mIuILATKgNrwRDdX8vOKet6qamN1STblxdYvb6bXE7+3iHuXePmfW4/ybk2703HMBCYt9ME+9yeA7cBx4HlVPSoiT4nIg8HDfgjkiEg1o100F4dgrgMOichBRi/SPqaqHeH+Jsx/2HGqlb/+1RHK8lL52Ip8p+OYGPDzigbuKvOSlZzAoz+u4J9er7Yb8SKMRFq/Wnl5uVZWVjodY1Y63NjFxmfeZUFOCn94WyEeW0zEzKC23iH++c1qMpLieWxdKZ+9s8TpSDFFRPaqavlE++zO2ChxpKmLT/9wD5nJCTz7mdutyJsZl5vq4aFVRbR0D/GLvY0EbD6ciGGFPgocPTta5FM9cWzZ9AHmZthNUcYZZXlpfGT5PI6d6+YfXj3ldBwTZIV+lttV08bGZ3aTHO9my6YPMD/bRtgYZ60pzeG2BVn84+vVPF/RMPkJZtrZ4OpZ7N8PNPGXvzhIcU4KP/rs7TaM0kQEEWHDLfkkJ7h58sVDZKUk8MFlc5yOFdOsRT8L+fwBvv7SCb6w5QC3FmXxwmNrrMibiBLncvG9T9/G8oIMnti8j921NuzSSdain2W+v6OGLRUNnG7rY1VxNh9dPo//c/ic07GMuUyKJ45nP3M7n3pmN5/50Xv8yx+Xc1eZ3fnuBGvRzxKqym8OnuU7r1XR2NnPJ28r5OMrC4hz239CE5k276ln+9Fm/lP5fDKTEvjsjyp49Viz07FiklWJWaC9d4jHN+/jz3+2n+yUBB6/ZxG3FmU5HcuYkKR64viTu0qYk57Ipp9U8r0dNTYvzgyzrpsI99Lhc3z510foGfTxV+uXkOaJx+2yWSjN7JKcMFrsK8508PWXTnC4sYu/+8RyMpLjnY4WE6xFH6EaOvr503+r5HPP7WNeZiK/+fM7+bN7FlmRN7OWJ87N0w/fypceWMpLR85x37fe5MV9jda6nwE2BUKE6RoY4ZmdNXx/Ry0uEe5dmsedi3KtwJuo8PDqImD0Tu4v//oIBxousKIwgz+5ayEfuWmuXXO6DlebAsEKfYTo6h/hp3vq+P6OGroHfawozOCBm+aRkWR/2procbHQw+haxi/sbeR7O2qobetjXkYiH1sxj48sn8ct8zNtoZwpskIfoVSVw01dbKlo4Ff7mhgY8XPf0jy++KHFHGzocjqeMTNi4+3zef1EC5vfq+etqlZG/Epuqod1ZbncvcTLnYtyyUn1OB0z4lmhjyAdfcNUnungvdMdbD92noaOATxxLjbcks8ja4q5MT8DwKZ5NTFpYNjPifPdnGzuobqll/5hPwLkZyZRNieVP7tnESuLMom3Lp7LWKF3iKpS39FPxZlOKs90UHGmg5rWPgAS3C4+UJrDx1bMo2fAR1KCzTZpzFgBVc5eGOBUcy9VzT00dPYTUPDEuSj1plI2J5XFeWk8/nuLnI4aEazQzxCfP8Cxc91UnOnkxX2N1LX30xtcNDkp3s2CnGQWZCezICeFgqwka5UYMwUDw35qWnupaumhqrmXCwMjACzMTWHdYi93L/ayemF2zK6PbIV+mnT1j3Co6QKVZzqprOtgf/0F+of9AGQlx7MgJ4UFOckU56TgTfPgsotLxoSFqtLaO0RVcy+9Qz5217Yz5AvgEliUl8rygkyWF6SzvDCDJXPTSfVEf/G/WqEP6bsXkfXAdwA38ANV/fq4/R7g34DbgHbgU6p6JrjvS8CjgB/4vKpuv8bvwzGBgHK+e5AzbX0cOdvFocYujjR1caa9HwAB5mUkcnNh5mirPSfFRssYM41EhLy0RPLSEnl4dRGDI37eO91BZV0nhxsvsONUC7/c13jp+FRPHLmpCawqyaY4N4XCrGTmZSQyLyOROemJUf/X9aQt+uDi3qeADwKNjC4W/pCqHhtzzJ8BK1T1MRHZCHxCVT8lIsuAnwGrgHzgVWCxqvqv9H7T1aJXVVRH+/18AWVwxM/gSICBET+DI356Bn109A3R3jdMe+8wHX3DNHYOUNfeR11HP8O+wKXXykyOpyAzafQrK4n5Wckk2opOxkQMVaV70EdT5wCtPYO09Q7T1jdE35Cftt6h9x0rMro6VlZyPBlJo1/pSfGkJ8bjiXeRGOcmMd5NYrwLT9zov4nxbjxxrvdt98S5iHO7iHMJ8W4XcW4h3jX678XHrmm8H+Z6W/SrgGpVrQ2+2BZgA3BszDEbgP8RfPwC8E8yOgh2A7BFVYeA08HFw1cB717LN3I17b1D3PWNNy4V84v/jn5N/fU8cS4yk+PJSfGwujib7NQEclI8zMtIJCUG/gw0ZjYTkUtFG9Lft29wxE/XwAhdAyN0D4xwIfjvwIif9t7RBt7AsJ9Bn58Rv+IP45KILgG3SxCCBV9GewREQBBunp/Blk13hO39LgqlYhUAY5eJaQRWX+kYVfWJSBeQE9y+e9y5BePfQEQ2AZuCT3tF5GRI6SNXLtDmdIhZwH5OobOfVWhm9c/pBPDz/3LNpy+40o5QCv1Ef2uM/1/clY4J5VxU9RngmRCyzAoiUnmlP6HMf7CfU+jsZxUa+zlNLJQrEI3A/DHPC4GzVzpGROKADKAjxHONMcZMo1AKfQVQJiIlIpIAbAS2jjtmK/BI8PEngdd19CrvVmCjiHhEpAQoA94LT3RjjDGhmLTrJtjn/gSwndHhlc+q6lEReQqoVNWtwA+BnwQvtnYw+j8Dgsc9z+iFWx/w+NVG3ESRqOmGmmb2cwqd/axCYz+nCUTcDVPGGGPCK7rvEjDGGGOF3hhjop0V+jASkfki8oaIHBeRoyLyBaczRTIRcYvIfhH5rdNZIpWIZIrICyJyIvh7Ff67aaKAiPy34GfuiIj8TEQSnc4USazQh5cP+AtVvQH4APB4cBoIM7EvAMedDhHhvgO8rKpLgZuxn9dlRKQA+DxQrqo3MTpoZKOzqSKLFfowUtVzqrov+LiH0Q/lZXcCGxCRQuCjwA+czhKpRCQdWMfoqDZUdVhVLzibKmLFAUnB+3iSsft13scK/TQRkWJgJbDH2SQR69vAXwGByQ6MYQuBVuBHwS6uH4hIitOhIo2qNgH/H1APnAO6VPV3zqaKLFbop4GIpAK/BP6rqnY7nSfSiMjHgBZV3et0lggXB9wKfFdVVwJ9wJPORoo8IpLF6ASKJYzOkpsiIp92NlVksUIfZiISz2iRf05VX3Q6T4RaCzwoImeALcDvichPnY0UkRqBRlW9+FfhC4wWfvN+9wOnVbVVVUeAF4E1DmeKKFbowyg4NfMPgeOq+i2n80QqVf2SqhaqajGjF81eV1VrgY2jqueBBhFZEtx0H++fHtyMqgc+ICLJwc/gfdhF6/exidXDay3wn4HDInIguO2vVXWbg5nM7PbnwHPBeaZqgc86nCfiqOoeEXkB2MfoyLf92FQI72NTIBhjTJSzrhtjjIlyVuiNMSbKWaE3xpgoZ4XeGGOinBV6Y4yJclbojTEmylmhN8aYKPd/AYymKDXnTR3GAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Why \"lognormal\"?\n",
    "\n",
    "sns.distplot(np.log(s2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Box plots\n",
    "\n",
    "<img src=\"figures/eda-boxplot.png\" width=\"800px\" heigth=\"800px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outliers, missing values\n",
    "\n",
    "An *outlier* is an observation far from the center of mass of the distribution. It might be an error or a genuine observation: this distinction requires domain knowledge. Outliers infuence the outcomes of several statistics and machine learning methods: it is important to decide how to deal with them.\n",
    "\n",
    "A *missing value* is an observation without a value. There can be many reasons for a missing value: the value might not exist (hence its absence is informative and it should be left empty) or might not be known (hence the value is existing but missing in the dataset and it should be marked as NA).\n",
    "\n",
    "*One way to think about the difference is with this Zen-like koan: An explicit missing value is the presence of an absence; an implicit missing value is the absence of a presence.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary statistics\n",
    "A statistic is a measure over a distribution, and it is said to be *robust* if not sensitive to outliers.\n",
    "\n",
    "* Not robust: min, max, mean, standard deviation.\n",
    "* Robust: mode, median, other quartiles.\n",
    "\n",
    "A closer look at the mean:\n",
    "\n",
    "$\\bar{x} = \\frac{1}{n} \\sum_{i}x_i$\n",
    "\n",
    "And variance (the standard deviation is the square root of the variance):\n",
    "\n",
    "$Var(x) = \\frac{1}{n} \\sum_{i}(x_i - \\bar{x})^2$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"figures/2560px-Comparison_mean_median_mode.svg.png\" width=\"400px\" heigth=\"400px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.992350413024771\n",
      "245.32716462110596\n"
     ]
    }
   ],
   "source": [
    "# Not robust: min, max, mean, mode, standard deviation\n",
    "\n",
    "print(np.mean(s1)) # should be 5\n",
    "print(np.mean(s2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.003974843024547\n",
      "148.8969352474893\n"
     ]
    }
   ],
   "source": [
    "# Robust: median, other quartiles\n",
    "\n",
    "print(np.quantile(s1, 0.5)) # should coincide with mean and mode\n",
    "print(np.quantile(s2, 0.5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions\n",
    "\n",
    "* Calculate the min, max, mode and sd. *hint: explore the numpy documentation!*\n",
    "* Calculate the 90% quantile values.\n",
    "* Consider our normally distributed data in s1. Add an outlier (e.g., value 100). What happens to the mean and mode? Write down your answer and then check."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>annual_salary</th>\n",
       "      <th>a_age</th>\n",
       "      <th>length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>7870.000000</td>\n",
       "      <td>9303.000000</td>\n",
       "      <td>9645.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>5.916921</td>\n",
       "      <td>14.266688</td>\n",
       "      <td>5.005694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>6.985214</td>\n",
       "      <td>2.902770</td>\n",
       "      <td>1.462343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.166667</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.083333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>4.000000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>180.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>15.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       annual_salary        a_age       length\n",
       "count    7870.000000  9303.000000  9645.000000\n",
       "mean        5.916921    14.266688     5.005694\n",
       "std         6.985214     2.902770     1.462343\n",
       "min         0.166667     1.000000     0.083333\n",
       "25%         3.000000    12.000000     4.000000\n",
       "50%         4.000000    14.000000     5.000000\n",
       "75%         6.000000    16.000000     6.000000\n",
       "max       180.000000    50.000000    15.000000"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's explore our dataset\n",
    "df_contracts[[\"annual_salary\",\"a_age\",\"length\"]].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Relating two variables\n",
    "\n",
    "#### Covariance\n",
    "\n",
    "Measure of joint linear variability of two variables:\n",
    "\n",
    "<img src=\"figures/covariance.png\" width=\"400px\" heigth=\"400px\">\n",
    "\n",
    "Its normalized version is called the (Pearson's) correlation coefficient:\n",
    "\n",
    "<img src=\"figures/pearson.png\" width=\"400px\" heigth=\"400px\">\n",
    "\n",
    "Correlation is helpful to spot possible relations, but is of tricky interpretation and is not exhaustive:\n",
    "\n",
    "<img src=\"figures/800px-Correlation_examples2.svg.png\" width=\"700px\" heigth=\"&00px\">\n",
    "\n",
    "See: https://en.wikipedia.org/wiki/Covariance and https://en.wikipedia.org/wiki/Pearson_correlation_coefficient.\n",
    "\n",
    "*Note: correlation is not causation!*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>annual_salary</th>\n",
       "      <th>a_age</th>\n",
       "      <th>length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>annual_salary</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.205404</td>\n",
       "      <td>-0.361611</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a_age</th>\n",
       "      <td>0.205404</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.430062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>length</th>\n",
       "      <td>-0.361611</td>\n",
       "      <td>-0.430062</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               annual_salary     a_age    length\n",
       "annual_salary       1.000000  0.205404 -0.361611\n",
       "a_age               0.205404  1.000000 -0.430062\n",
       "length             -0.361611 -0.430062  1.000000"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_contracts[[\"annual_salary\",\"a_age\",\"length\"]].corr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a21219da0>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEGCAYAAABhMDI9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZycZZXo8d+ptauXJJ1OJ4R0MAFDIGAIJECAGUEYEWQT2UQCikKIiKhXERyHAWW8FwTGURRCouyIQZCL4gaDonNZhIRNBAKymQ4x6XQWeqmu9dw/akm9VW91qqqrUtXp8/18+tPdb1c9daqTfs/7Pst5RFUxxhhjMjz1DsAYY0xjscRgjDHGwRKDMcYYB0sMxhhjHCwxGGOMcfDVO4CRmjRpks6YMaPeYRhjzKiyatWqjara6fazUZ8YZsyYwcqVK+sdhjHGjCoi8k6xn1lXkjHGGAdLDMYYYxwsMRhjjHGwxGCMMcbBEoMxxhiHUT8raSSSSaV3IEo0niDg89LREsDjkXqHZYwxdTVmE0Myqaxe38f5d6yke3OYrvYQy89ZwOwpbZYcjDFj2pjtSuodiGaTAkD35jDn37GS3oFonSMzxpj6GrOJIRpPZJNCRvfmMNF4ok4RGWNMYxiziSHg89LVHnIc62oPEfB56xSRMcY0hjGbGDpaAiw/Z0E2OWTGGDpaAnWOzBhj6qumg88icgtwPLBBVfdNH1sBzE4/ZAKwRVXnicgM4BVgdfpnT6nqklrF5vEIs6e08cCFh9msJGOMyVHrWUm3AT8A7sgcUNUzMl+LyPXA1pzHv6Gq82ocU5bHI3S2BXfUyxljzKhQ08Sgqn9K3wkUEBEBTgeOrGUMxhhjylPPMYZ/Btar6us5x2aKyHMi8kcR+ediTxSRxSKyUkRW9vT01D5SY4wZQ+qZGM4E7sn5fh2wm6ruD/wv4CciMs7tiaq6TFUXqOqCzk7XfSaMMcZUqC6JQUR8wMeBFZljqhpR1d7016uAN4A96xGfMcaMZfW6Y/gX4FVV7c4cEJFOEfGmv94dmAW8Waf4jDFmzKppYhCRe4Angdki0i0in03/6BM4u5EAPgi8KCIvAPcBS1R1Uy3jM8YYU6jWs5LOLHL80y7H7gfur2U8xhhjtm/Mrnw2xhjjzhKDMcYYB0sMxhhjHCwxGGOMcbDEYIwxxsESgzHGGAdLDMYYYxwsMRhjjHGwxGCMMcbBEoMxxhgHSwzGGGMcLDEYY4xxsMRgjDHGwRKDMcYYB0sMxhhjHCwxGGOMcbDEYIwxxsESgzHGGIda7/l8i4hsEJGXco5dKSJrReT59MdHc372dRH5m4isFpGP1DI2Y4wx7mp9x3AbcIzL8e+q6rz0x68BRGQO8Algn/RzbhQRb43jM8YYk6emiUFV/wRsKvHhJwE/VdWIqr4F/A04qGbBGWOMcVWvMYaLROTFdFdTe/rYNGBNzmO608cKiMhiEVkpIit7enpqHasxxowp9UgMNwF7APOAdcD16ePi8lh1a0BVl6nqAlVd0NnZWZsojTFmjNrhiUFV16tqQlWTwHK2dRd1A9NzHtoFvLuj4zPGmLFuhycGEZma8+3JQGbG0i+AT4hIUERmArOAp3d0fMYYM9b5atm4iNwDHAFMEpFu4ArgCBGZR6qb6G3gAgBV/auI3Au8DMSBz6tqopbxGWOMKSSqrt34o8aCBQt05cqV9Q7DGGNGFRFZpaoL3H5mK5+NMcY4WGIwxhjjYInBGGOMgyUGY4wxDpYYjDHGOFhiMMYY42CJwRhjjIMlBmOMMQ6WGIwxxjhYYjDGGONgicEYY4yDJQZjjDEOlhiMMcY4WGIwxhjjYInBGGOMgyUGY4wxDpYYjDHGOFhiMMYY42CJwRhjjENNE4OI3CIiG0TkpZxj14rIqyLyoog8ICIT0sdniEhYRJ5PfyytZWzGGGPc1fqO4TbgmLxjjwD7qupc4DXg6zk/e0NV56U/ltQ4NmOMMS5qmhhU9U/AprxjD6tqPP3tU0BXLWMwxhhTnnqPMXwG+E3O9zNF5DkR+aOI/HOxJ4nIYhFZKSIre3p6ah+lMcaMIXVLDCLyDSAO3J0+tA7YTVX3B/4X8BMRGef2XFVdpqoLVHVBZ2fnjgnYGGPGiLokBhH5FHA8cJaqKoCqRlS1N/31KuANYM96xGeMMWPZDk8MInIMcClwoqoO5hzvFBFv+uvdgVnAmzs6PmOMGet8tWxcRO4BjgAmiUg3cAWpWUhB4BERAXgqPQPpg8C3RCQOJIAlqrrJtWFjjDE1U9PEoKpnuhz+cZHH3g/cX8t4jDHGbF/JXUkism8tAzHGGNMYyhljWCoiT4vIhZnVysYYY3Y+JScGVf0n4CxgOrBSRH4iIh+uWWTGGGPqoqxZSar6OvBvpGYVHQ58P1336OO1CM4YY8yOV84Yw1wR+S7wCnAkcIKq7p3++rs1is8YY8wOVs6spB8Ay4F/VdVw5qCqvisi/1b1yIwxxtRFSYkhvfBsjare6fbzYseNMcaMPiV1JalqAugQkUCN4zHGGFNn5XQlvQM8LiK/AAYyB1X1P6selTHGmLopJzG8m/7wAG21CccYY0y9lZwYVPWbtQzEGGNMYyg5MYhIJ/A1YB+gKXNcVY+sQVzGGGPqpJwFbncDrwIzgW8CbwPP1CAmY4wxdVROYuhQ1R8DMVX9o6p+BlhYo7iMMcbUSTmDz7H053Uichypgeiu6odkjDGmnspJDP8hIuOBrwA3AOOAL9ckKmOMMXVTzqykh9JfbgU+VJtwjDHG1Nt2E4OI3ABosZ+r6sVVjcgYY0xdlXLHsLLSxkXkFuB4YIOq7ps+NhFYAcwgNbPpdFXdLKkNoL8HfBQYBD6tqs9W+tpmdEkmld6BKNF4goDPS0dLAI9H6h2WMWPSdhODqt4+gvZvI1WV9Y6cY5cBj6rq1SJyWfr7S4FjgVnpj4OBm9KfzU4umVRWr+/j/DtW0r05TFd7iOXnLGD2lDZLDsbUQTn7MXSKyHUi8msR+X3mY7jnqOqfgE15h08CMsnmduBjOcfv0JSngAkiMrXU+Mzo1TsQzSYFgO7NYc6/YyW9A9E6R2bM2FTuArdXGPkCtymqug4g/Xly+vg0YE3O47rTxwqIyGIRWSkiK3t6eioIwTSSaDyRTQoZ3ZvDROOJOkVkzNjWSAvc3PoMXAe9VXWZqi5Q1QWdnZ1VDMHUQ8Dnpas95DjW1R4i4PPWKSJjxrZyEoNjgZuI7E9lC9zWZ7qI0p83pI93A9NzHtdFahGd2cl1tARYfs6CbHLIjDF0tNj2H8bUQz0WuP0C+BRwdfrzgznHLxKRn5IadN6a6XIyOzePR5g9pY0HLjzMZiUZ0wBqusBNRO4BjgAmiUg3cAWphHCviHwW+DtwWvrhvyY1VfVvpKarnltqbGb083iEzrZgvcMwxlBe2e3vAP8BhIHfAvsBX1LVu4o9R1XPLPKjo1weq8DnS43HGGNMbZQzxnC0qr5HasFaN7AncElNojLGGFM35SQGf/rzR4F7VDV/fYIxxpidQDmDz78UkVdJdSVdmN7Rbag2YY0etSjlUO02rdyEMaYc5Qw+XyYi1wDvqWpCRAZJrVYGQEQ+rKqP1CLIRlWLUg7VbtPKTRhjylVOVxKqullVE+mvB1T1Hzk/vqaqkY0CtSjlUO02rdyEMaZcZSWG7Rhzl5+1KOVQ7Tat3IQxplzVTAxF92zYWdWilEO127RyE8aYclUzMYw5tSjlUO02rdyEMaZcklpXVoWGRH6uqh+vSmNlWLBgga5cWfFeQiNms5KMMaORiKxS1QVuPytla89hT/aq+vP05x2eFBpBLUo5VLtNKzdhjClHKdNVTxjmZwr8vEqxGGOMaQClbO1pxeyMMWYMKWflMyJyHLAP0JQ5pqrfqnZQxhhj6qec6qpLgWZSJbd/BJwKPF2juMY0G3w2xtRTOXcMh6rqXBF5UVW/KSLXY+MLVWclMYwx9VbOOobM8tlBEdmV1FafM6sf0thmJTGMMfVWzh3DQyIyAbgWeJbUjKQf1SSqMcxKYhhj6q2c6qpXpb+8X0QeAppUdWttwhq7MiUsck/m1SiJUa32jDE7v5K7kkTknMwHcAZwUvrrsonIbBF5PufjPRH5kohcKSJrc45/tJL2RzMriWGMqbeSS2KIyA053zaR2rf5WVU9dUQBiHiBtcDBwLlAv6peV+rz610Soxbi8SQb+iPEE0l8Xg+TW4P4fJWXtbJZScaYfCMqiZGhql/Ia3Q8cOcIY4NUgnlDVd8RsZNVMqm83tNf1VlEVhLDGFOOkVRXHQRmVSGGTwD35Hx/kYi8KCK3iEh7FdofVWwWkTGm3soZY/iliPwi/fEQsBp4cCQvLiIB4ETgZ+lDNwF7APOAdcD1RZ63WERWisjKnp6ekYTQcGwWUXUkk0pPX4S1mwfp6YuQTI657UKMqVg501Vz+/3jwDuq2j3C1z+W1DjFeoDMZwARWQ485PYkVV0GLIPUGMMIY2goNoto5GxRnzEjU84Ywx9r8PpnktONJCJTVXVd+tuTgZdq8Jp1U8ogcEdLgDs+cxDv9A7SHPAyGE3wvo7mhppF1Oh7UBTrjnvgwsNsrMWYEpRTK+njwDXAZFL7OwugqjqukhcWkWbgw8AFOYe/IyLzSC2eezvvZ6NaOVexkXiSyx98yfG4RlGLq/Fqt2ndccaMTDmDz98BTlTV8ao6TlXbKk0KAKo6qKoduYvkVPVsVf2Aqs5V1RNz7h5GvVIHlRt98LkW8VW7Tdvn2piRKScxrFfVV2oWyShV6iBnqVexjX61W4v4qt2mLeozZmTKGXxeKSIrgP8LRDIHM1t7jkXldIGUOqjc6IPPtYiv2m16PMLsKW08cOFhtqjPmAqUs/L5VpfDqqqfqW5I5annyueevggn3/h4wQnNbZCz1CTS6DNqkknl7d6BgsHxGR0tIxpjWLtlkEhc8QgkFYI+YdqE5oZ4z8bsjKq18tm2+MxTThdIqVexo+Fqt9qD48mksjUcZ8ldq7JtLl00n6njtKHetzFjRTkL3DpF5F9FZFl6VfItInJLLYNrdOUOcmZKU0xrb6azLVj0pFfq4+qhFoPPG/oj2aSQaXPJXavY0B/ZzjONMbVQzuDzg8B44L+BX+V8jFljcZCzFoPPsUTStc14Illxm8aYypUz+NysqpfWLJJRaDR0+1RbLQaf/V6Pa5s+70hKeRljKlXOX95DY3F/hO1p5G6fWqjFXdLk1iBLF813tLl00Xwmt9oqZWPqoZxZSX1AC6mpqjFGuPK5WnbG/RhisURqP4ak4vMIk1uD+P2NMV0VIBqN0zMQzcbX2RIgECjn5rNQJBJn4+C2Nic1BwgGK2/T9qAwZnjVmpXUJiITSZXabqpWcMYpFkvw6oZ+PpczQ+emRfPZa3JrQySHWCzB6p6BqsYXiyV4bWP12mz0Kb/GNLpyZiWdB/wR+C1wZfrzv9cmrLFrQ38ke4KE1CDs5xpohk4t4qt2m41eVsSYRlfOGMMXgQNJldv+ELA/sLEmUY1h8aS6z9BpkP0EahFftdts9LIixjS6chLDkKoOAYhIUFVfBWbXJqyxy+cR17URvgbpAqlFfNVu04roGTMy5SSGbhGZQKpW0iMi8iDwbm3CGrsmtwa5KW+Gzk0NNEOnFvE1Bz3ceNYBjjZvPOsAmoOVTVcdi+tLjKmmkmclOZ4kcjipxW6/VdW6dtzWe1ZSLWa/NPqspGrHt3bzIN/779c5/4O74/UIiaSy/E9v8sV/mcW09uaK2rRZScYMryqzknLVaDe3uiv3ZFKr2S9+v7fiE+KO4PV6Ut0y6d+Td4QL0QI+L0+82cu9q7btFNvVHuIS314Vt5lZX2KMKZ8tLU3LnORPvvFxDrvmD5x84+OsXt837CbyGwcirrNfNg40xgyiWqjk97Q91vVjTGMZ2aqknUgl+wQPxRJ0tga5/Pg5TAj52RKOsfSxNxiK7bw1fmqxn7LHI8zqbOXeCw4hlkji93qY3LrzryI3plFZYkirZIpj0Ovha8fM5pL7Xsx2JV176lyC3p33hFaLqaDJpPJ6T78tSDOmQdStK0lE3haRv4jI8yKyMn1soog8IiKvpz+376h4KpnimFSySQFSJ8hL7nuRkS45iMeTvLslzDu9A7y7JUw8PrI7kGg0ztrNg7zTO8DazYNEo/GK26rFVNBaLEgrdctVY0yheo8xfEhV5+WMjF8GPKqqs4BH09/vEJX0cyfUfWFWsoKZXhnxeJJX1/dx+s1Pcvi1j3H6zU/y6vq+ipNDNBpndc8AZyx7isOvfYwzlj3F6p6BipNDe8jvWvCuPeSvqD2ASJXvQmoxDmLMWFLvxJDvJOD29Ne3Ax/bUS+cW0L78Us/xAMXHrbdrgyPuC/MEqm8+6Pam9b0DERdy030VHg1vjkc4/uPvsblx89hxeKFXH78HL7/6GtsDscqag9S1Rjdfo+VspIYxoxMPccYFHhYRBS4WVWXAVNUdR2Aqq4TkcluTxSRxcBigN12261qAZU7xdErcM0pc7n0/m1jDNecMpeRDDFUe9OaWpSbePjlDTz88gbH8StOqHyMQYr8HisdXrCSGMaMTD0Tw2Gq+m765P+IiLxa6hPTSWQZpBa41SrA7fF4PNz+xFuOWUm3P/EW3z55bsVtVnvTmky5iYL2Rlhuopob9YC4/h6vPHHfBorRmLGjbl1Jqvpu+vMG4AHgIGC9iEwFSH/eULyF+utoCfDlD8/mqode5oxlT3HVQy/z5Q/PbqhNazpbAq4lLDorjLFWG/VcfNSejt/jxUftWfF7tnURxoxMRSUxRvyiIi2AR1X70l8/AnwLOAroVdWrReQyYKKqfm24tmpdEmO41dDJpPJ27wDv9A7SHPAyGE3wvo5mZnS0jGiaZTyeTJWcSCTxpef0+3yV5/DRsAlOtctsWEkMY4ZX9ZIYVTAFeCA9SOsDfqKqvxWRZ4B7ReSzwN+B0+oUH7D9khe9A1HOueXpgi6LkSz2AvD5POw6ofLB11zJpPLmpsGqrhGodrmJZFL528aBho7RmLGkLl1Jqvqmqu6X/thHVb+dPt6rqkep6qz05031iC9je7NbRsMg52iYoTMaYjRmLGm06aoNZXsnfqnBdNVqGw3JazTEaMxYYolhGNtb5ZuZrpo7yDnS6arVNho2rRkNMRozltRl8LmaajX4nEwqW8JR1m0Z4oKcTepz+757+iL86oVujpwzlaQqHhF+//I6jtuvy7V/OzOonFsozm1QuZqDxcmk8vbGAd7ZlDNAPrGZGZMqHyAfGorTG94WX0coQFPTyAazNw4MEY0riaTi9QgBnzCppaniGG3w2ZjhNeLgc0PLHXTubA1y1Un7MnNSC81BL5NatlX9nNDkY/7MSXxy+VPZxHHTovlMcDlJZkpdLMlJMksXzWevKW2O5BCJxHlt40B2tXKmzT0ntVSUHOLxBIOxBJc/+JLjdePxBIFA+e0NDcV5vbcwvlkdLRUnh2g0wfr3ogVtjvP7K2qzVvtkGDNWWFeSi9zB0OfWbOHc255h0Y//jCCOE0vPQJQb8spD3PDoa67lJkotdbFx0L2ExcbBygZiewairq9baUmM3rB7fL3hygeKq92mDWYbMzJ2x+Ci9MFQ5VOHziwo5SAUds+VWuqi2iUsGr29WrRpg9nGjIzdMbgodTBUlWxSgNTJ59L73ctuZ0pd5LeZX+oiU8Ki4HEVdoE0enu1aNMGs40ZmTGbGIar11+spEJ7yO94jgKdrUFuPns+KxYv5Oaz59NZpIxDqaUuQgGPawmLUKCyf6qql8QIubfXEaq83MS4kPt7Hheq7D1bSQxjRmZMzkoqZXAyf1ZLe8hfsMvYzy5YyNu9gwU7uM3epY2JLcVnJQ1X6qKnL8JTb2xg//d1ZGfoPPdOLwv3mFzRSt5azPip9qykDX1D9PaFGRcKZtt8Lxyhoy3E5LamsturVakSY3YmNispTyn7FueXVOjpixQ8551NYdcd3H7+uUNdX7eUUhcTmnzM6BzHJ5Ztf6ZTKbaEo7yxYaAgefl28bgmr+2Jx5P8rXdgu7OryuH1wJZwgvPvfMoR4+TxFTVXs1IlxowVY7IrqZLBSbfnSPp5+e0MjWArzmpvrBOOJlyTVzha2UBstTcSAhiIJPjOb1c7Znd957erGYhUuGudDT4bMyJjMjFUMjjp9pzBaMK1ndGwUU+iwhk/1Y4PwCtCZ5uz/7+zLVDx73E0lCoxppGNycRQyeCk23P2mNzCTWcd4Dh241kH0BKsfPZLqbOXSlVsxo+3wr72ascH0BL0ctGRsxz7MVx05KyKf4+joVSJMY1sTA4+Q2UlE/Kfo6r8+sW1HDlnKqqKpEtifHS/aRUNmkKqD39rJMpQNJkdiG0KeBgfDFTUh9/bP0R/JE4sAR6BpILfC61BHx2t5ccYjyd5e9MAazaFswO70yeGmDGxpeIxhg19Q/z6hbUFpUUq/T329EV45d3N7DF5XPZ3+MaG99h71/YRjTGUWtLEmNHABp9dVFKvP/85G/uGOGBGh6Mkxo1nHTCi27B4PMm7WyIF5SFaOnwVnYQ8Au+F43zu7me3tXfWAYyrcDBbVRmKJQtKbIzkAsPnwbW0SKXn3HEBL+2tIc7IG8AfF6j8Tq7UkibG7Azsf/QIROJJfvXCWm799IH8/iuHc+unD+RXL6wlMoLB52qXhxiMJrNJIdve3c8yGK0sxloMPoejSdf3HK4wxmqXFYHavG9jGtWYvWOoBr9XOGFeF+fe9ozjKtI/gs7sRi9hMRpKYtQixloMuhvTqOyOYQTiSVyvIovdMKS6icK80zvAu1vCxF0e2OglLEZDSYxaxFiLQXdjGlVd/leLyHQR+YOIvCIifxWRL6aPXykia0Xk+fTHR+sRX6nKuYrM9FGffvOTHH7tY5x+85O8ur6vIDm0Bt3LQ7QGK/unCgU83Ogyc6rSEhuTW4Ou8eWX9ihHwOceY6DCvvtqlwGB0kuaGLMzqMusJBGZCkxV1WdFpA1YBXwMOB3oV9XrSm2rVhv1lKJ782B2hXJGV3uIny5eSFd7s+Ox724Jc/rNTxY89t4LDnGshl67eZCX393KnF3HZ2fUZL6fltdmKdZvDbMlHMXr8WZnJSWSCSaEAkwZP/wq7GJqURKju7efKeObs22u3zpIV0drxbO7arGZ0Notg0Timv09Bn3CtAnNVmbDjEoNNytJVdcB69Jf94nIK8C0esQCzmmofp8Hn0cIR4tPY808PqnK3ecdzLd/9TIPv7xh29W430NPX8QxFbbUu4tQwMMu4/Nm1IzgCn8onuTH//M2539wdxBBVfnx/7zN5498f0XtRSLuG/VUupEQQJMf/H5/wSyiJn9FzdUkxt6BKGcu/7OV2TBjQt0Hn0VkBrA/8GfgMOAiETkHWAl8RVU31/L13QrqXXvqXL7z29X09Edci+vlP/6HnzyALxw5i3e3DvGD37/OlSfsw+nLnqKzNcjFR81i5qQW/F7h6DmTefjlDdnX7moP4c/rox6KJnkoPdPJ6xESSeW+lX/nU4fOhJby31+Tz8PJB0xzDJBfe+pcmirspik242fF4oVMq/Ck2xd2n5W0YvFCxlVww1CLGK3MhhlL6jpyJiKtwP3Al1T1PeAmYA9gHqk7iuuLPG+xiKwUkZU9PT0jisGtoN4l973IkiP2cN35K//xna1BNg1EGRfys/ukFiaEAsSSSmdrkK9+ZDaXP/gSR1z3GGcse4qLjpzF0XMmA2RP0AG/827E5xWO2y91Ij/y+j9y7m3PcNx+0/BVONMpoepaKylRYRdiuTN+hitvXmmb1Y6xFLbHgxlL6pYYRMRPKincrao/B1DV9aqaUNUksBw4yO25qrpMVReo6oLOzs4RxVHsSnBCyJ/9OveqMPfx+0+fkD35H37tY6ktQA95Hy0BL0uO2KNgE58L736WSz6yl6NQ3FDeXP1YQrkwb93BhXc/SyxRaW2jIifJCtsrZ8ZP5u7q5Bsf57Br/sDJNz7O6vV9BclhNMxKsj0ezFhSl64kSVUz+zHwiqr+Z87xqenxB4CTgZdqHUvmSjC/73hLOJb9OveqMPfxS47Yg9ufeIvLj5/DhJCfLeEYP/j961xxwj5MCPldT8ibBqKcseypbNv+vC6dePpuI7fNpY+9UfHVrj99ksx/f5WeJMeFvPxsyULiidTdiFcEnxdaXeoa9Q5E+e4jqx3v5buPrObbJ8919Ms3+VOzki7MWZ1941kH0OQvvG4ppZRJOTGWyuMR9uhoZsXihdkB7c4SyqgYMxrVa4zhMOBs4C8i8nz62L8CZ4rIPECBt4ELah1I5krQbYzB7aqwPeRn6aL5LLlrFbuOb3Ld89kj2yqv5p+QB9PlrrP7IuSdWEI+D187ZnbB/gmhCscEWppS018LSmw0VdaeHy8b+4cK2msPOq+ck0klFk9w6bF78/feQa7+zav09Ee45pS5JJPOuyQRGN/s57ZzD9pWz8kn5BdDLWWDpXJiLEcslmB1T+GA9l6TW/H7rTvJ7FzGbBG9XOXMSurpi/CNB17klPnT2XfXcdmZNBld7SFWLF7I3zcN0hr0OWoU/dcZ8+hsC2ZLXm8NR5nYEqQ54Mu+TqnTWku1dvMg3/zlXzll/vTsVfv9q9ZwxQn7VDT9de3mwaLvOdOe2wn8mlPmct3vUgP6+e/l3c2DXOkS45Un7MOuOTH29EU4+cbHtzszqJQYa/G+jRlNGm66aqNxLahXZAZQNJ7g4Zc38PDLG/jTJUe473WgypnL/8z+0ydku1Gmt4fo6Y9w1o/+7LgT+NJPn3fMfooXm9aarHw/hky8ub5x3JyCx5bSTRNPKofu3sH5H9w9O2tq+Z/edHR1uQ3oX3r/i1x+/BwuuHNVQcG9WJEY/y0vxlJnBo2Gsh3GNDJLDEVEo3F6BqJ5/cmp7pfff+Xw7P7Jbt1F3nQfyHNrtnDBnavS+0Mfwud/8lzB7KDMyTKztehwbeavjUgmdbtloH2e1DTZ/Kvx/C6sUrtpWgJezvvgTLo3byu7fd4HZ9KSU7k0f4B+yRF7MCHkZ3JbkKPnTC6YyeMr9p7zYq54B7wAABswSURBVCw2HlRqe9Uo21HNNo1pVFboxUU0Gmd1zwBnLHuKw69NTTV9rWeAtzeljmWmkYL7hjAeDwWzVxJFrjjzZz9JkU1mRHDO7vlHH29vGthuiY0JIQ9fOGpPxyY4XzhqTyaEnP/0xfbB7s3bUjSZhN7+KJc/+BJnLHuKyx98id7+KLk3NJkTeGbWVua1z77laS4+ak/aQ86Vax5P8d9jrlJnBnWE3EtidIRGVhKj2qVAjGlUdsfgwm3f5SV3reKqk/Z1HHttfT/3PP2OY9bN7U+8xRUn7MMDFx5GMpkkoWS7TkqZ/RSNJwpmOmXadJy071zJdaft53jc9x99jStP3NfRf79lmMVjLTmLx0rtpokmk9z6uDO+Wx9/i38/YZ/sYzIn8H9sHSqYsrvkrlUFYwLJJPxp9fqCRX0zOmY6XtvjEWZPaeOBCw8btrurNxzlhkdfc8R4w6OvpcZVKiyL4fd7mdXR4piV1BEK2MCz2SlZYnBRrD+5OW+jl9/8ZR1fOGrPgpkqrUEPbU0B3u4d4J3eQZoDXnYZ38QPPrk/F6W7k4rNfgpHI3zj+DnE0jV5OlqDfOP4OYzPu8Lv3hxm2oQmwulZTgGvhws/9H4EZ593Ut2nvybz+vlL7abxCq4zsXLX32VO4C1Bb0nJxu8VTlkw3dE9dcqC6a7ly0vZYClR4phFOWKxhGuZDZuVZHZGlhhcFOtP7mgNsmLxwuzJ9dgPTGXVWxv5yfkLHVtSdrTsSkKjrH9vyLHT2U1nzefu8w7mvXCMreEYPo+H731iHqGcWUnhGLw3GCvYca0l7+Rz9JzJbB6MOdq/9tS5ePLmeAa87tNfA3mlONym7bp10ySVgruAS+9/kRWLFzoe5/EIIb+vpGSjuq17KjfG8RUWSyo6TjOC8YAN/ZHiZTZsVpLZyVhiyBOPJwkF3Of+f+e3r2SL5V176lx2n9RCZ1vQsSXl0kXzCfpS0139Xg+XHz+HpY+9wXNrtnDD71/ja8fsTXPQx0A0QXuLv6C0dCTuvuPaisULsye7rvYQ3zhuTnaGU+Zxl9yXOkFnBqn9Pg/xpLp2/Vxxwj4Fg9mldNMUGytxK7HR0RLgnvMPLqhImp9sYknlsVcLu5KmHzqzoM1SZMZp3NaXVKoWs5Iq2Xd8R7Znxi5LDDly9/U9Y34X96TvBLwe4Vu//Gu2ayJzEr73gkMKNur5/qOv8cWj9uSCnKRyzSlzefC5tZwyv4tP3/p09vj3PjGPS372onO66jAnoKtO2pcJzX7Gh/xI+nj+45KqnHzj49kCfrN3aS26CC+zJqCrPcTNi+azy/ggsXiShCqK+wlvezOxciWTytZwvGCf5Knj1HHCCuTUh8pd+RyosD5UUnEdp8kdBylXtWcllToLrF7tmbHNFrjlKLa47K7PHswR1z1W8Pg/XnIEh1/rPH7z2fO56qGXC9q4/TMH8albni44nirT7cXrEYLpu4c7nniLUxfs5rh6PufQmRx2zR+4+ez52QVqxRau5S6uW7F4YdGFWYdd84fssaPnTOYLR85ydGEtP3sBsya3sjkcy16FxpMJ3uwZLEg0u3c2M3V85XtQlLoILx5PbneK7vqtYTb2R9jQF82OWUxuCzCpNVjxHhQb+4ZYt3WooItv6vgmJlWwZ0Spi/Xq1V6G3YXsvGyBW4mK7ZmgaNE+6/zjHS0B1za8Iq7H25r8nP3jbYvefv3FQzntwN1Ys2nbQOxpB+7GuJCHm8+ez55TWrns2L1p8nu46MhZBfWF/F5hKJbkkvuezd5BFLuzyPW5I/ZgY3+U60/bLzuGcv6dK7nn/IWcmdNVtmLxwqKzpsB5Iil695O3B4WnyIB2/vkn944u9w5kryltjuTQ5BcUcYxZLF00nyb/CLppUNpCPkfZDp83dbwS1S7jXYuy4HYXMnZZYsiR2dc3PwH8Y+tQQZ/1TWcdgN8jXHvqXMfA7sSWQLaNzOKujpZA0f0Y3t444OiKGoqkylTnD8S2BX3ZO5Gu9hB3n3ewaxXWFYsX0hzYNhvI63F/T96cRQL7T5+A3+vl8gefp3tzmKPnTOb60/dja3oqbWdrkO7NYbo3hxGBz/7T7nzlZy9kY7n+tP3wSOqqOglE40kS6SmdxRJqT18ke/VZrOvniryun57+iOse2z+74BCm5tyB9EeS/PL57oIxi3MOncn4CseJY3Fl0Y8K7/jyB91LJeL+uxGXLrlSlDqrrBzF1rbY5kQ7P0sMOTL7+i7JGx/4zm9XA2RPXLuMbyLo8xBLJpnUFuSqk/bNXt17vcLNZ8/ne//9WsFV8E2L5gNkB7CXLprP5f/XWUA2mnTfP+Gnixc6jvX0RYreCew6IZQ9SYT8UvCeli6aT8i/7cR08VGzsj/ff/oEPnXoTM655WnH7+C6363muTVb8Igwodl55ZxIJhARkijxhBJLpBLD5oE4t557IOfe+ozjd+D1QCSeYN17Yfxez7B3DLkD5NEid3SxvDsQv1c4Pm/M4qazDsDvKzzplroFaLUHn70C15+2X0GCrXBYpeRZZeWwzYnGLksMOTweoTngzZ7oO9uCXP2bV3huzRYALrhzFUfPmczFR+3Jtx99jW+dtC8+T5L3T27NlneOJxNMaQty6bF7O8YUMtMbb/30gXz2n3ZnSzhGOJqgpz/iiKHorJ9k/roD9zsBn8dD9+Zw9qQTjStNfnGcyCFJNK5ce+pcWoM+xuWUCHfbR+LS+1/k1k8fyNfuezF1Ba7w7qbBbDKcMj6I1yNsDcfo6Ys47qBuOHN/7l9yCOF4Eq8IAa+weTBKwOdNX80n8IoUnQKbO0B+z/kLXct75HdrxBJadGZXrqGhOOFk3HEsnIzDEAXJoZzB51L65f0+D61Br+OiojXozZZhL7dvv9TFf+Wo9l2NGT0sMeToHYhmr5QB7jn/YM49bCYvr+vLnpwuO3bv7GO+dRL8Y2uksET2JF/RWUO5+zHsP30CN511gGNAs9j+Cf68P/DOtiDfPX0/vnzvtivO756e6tL58r3P84NP7s/VH/8AAD190YIY24J+2lv89IXjbOyPcvScyZxzyAy62psdU2wzcXtEuPLEOagqW13WT0xqCbBmUzh7PPO8mx77G187Zm96+yMMRhNMnxgillDOu+PpnBP+wUXvfnLbisQTrgsKmwOFe1ps7+o+mVSixOneHClo730d0JT3pzEu5OHWcw+kO2fsp2tiiHF5Cw+TSXUsbByMJnhfRzMzOlocJ+l4QnnmrV6OnDPVsQbmo3OnVdy3X8riv3J4i0z7rfSuxowelhhy5N86e0S4+jevcuunD2RrOEbvQJSt4Vj2MfEi3T73nL+Q1zf0D1sCA1J95h2tAe74zEGICCF/quS32xqK3DuErvYQqorf53Fccfp9HhLpk+lFP3mOy4+fQ6zIOoZ/P2EfonFl+f+8yYRQgIuP2tPR3XT9aftx9W9e5bk1W+hqD6VnOnmJJop3deWObUC6gN7h73dM0b3hzP2JJ5wnfJ/HU6TQn/Ok6/VI0VIXudVwS7m6T12Nl77X9GAk1dU0fWJz9s7L60kdz52UtDkcKVjYeO2pcxnf7KMjpwaJ1wtHf2AqiXSvjJD63uutvG+/lBlb5fB4PK5jP98+eW7FbZrRwRJDjvwBvC3hGD39Eb5234t89SOzuX/VGi75yF7ct+SQbD+6+25rSZY+9kbhgPWi+dzw6GsAOTNlPJx+81PZTXu6JoZoDXqyayg86e6pWDLJ5cfPoaMlwJRxTaiSLa+RkRkM3X/6BJ5bs4UJIf+w/fcX3v0slx8/J7UVad4J8is/e4GrP/4BLvv5X7jprAPoj8RZ/j9v8o3j5rhf3Se1YHOib560D796YW3BIPDZeQvXmoPificQLFzF/Y3j9sYjqQS464QQex23N5Iei8j0p48vcnWfW1Zk2FlTLuMGPq8QG0qydvNQts1p7U348rqcwtFk0cSZm7wE6O2PFbznaROCFfXtlzpjqxwdLQEuO3Zv3ukdBFK//8uO3du2Mx0DLDHkyB/Au3/VmuzV+4PPreWiI2dlBzRTJzvcd1vze3luzRau+11qW8s9J7fi93l4YFU3l3xkLy47du/sPgafP/L9XHXSvnS1N3Pm8qe4f8khJDW1AtojEFNFBDwIF9y5CoDHLjkCcO+qGowm+OpHZnP7E28xGE2gw5Sw6N4cZpdxTUVrGk2f2Mxdnz2YL694Prv7mt8jXPDPMwrWWfg8wsQWv2OW1i7jgiUtXBsYcr9yvzdvtXdzwEP3lhifu+tpx8m0a4KP59dsZfrEEOOa/KDKYCThuGr/4ScPYDDiIxyLpKvYCn6Pe2FDv0dIJrWg68fvlYLxpHhCHQPkxcaI8ve5DkeTruVUJu4ztaIZRhuKzNiqdIOnjFDA43jPI5jkZEYRSww53Abw2kP+7Pe5C8WaA14SSYpfHZLaj+Gqh17mnvNT369Y1c31//169vW62kNcfNQsJrYEiMRTM258XmEgkihYn5Apz51ZZewR95MapBLB3ecdzIb3IsOuY8hMr40nk3S1h+hsDWb3ThiMJvB6IJHEcSzo93D8vC7njJ9F8wkGPFz5i5f52jGzufMzB5FQJZbQolNqc0/4w12553bjDUaTRbuSLn/wJa49dS6bB2KpRPCH1x2P++EfXuebJ+7LaenB7NSU3A+43lm0hTz8fdMgzUEvk1qCeDyC3ycM9CdZcte297100XzGh4TL7k/t6NfREmDX8U3uySbvqr0p4GH+zEmOcio3LZpPU8BDe6j8GUbxRNL97jVR2QZPAH2RCBtd7mqaAx7Gh8pf1Geqp9YLDy0x5HEbwOtsC7J286Djj31LOEZHen5/rtyuiEy3zVUP/ZX/+NgHCtY8XHvqXNZsHszWLupqDzEUSw57Mr3xrAO468m3+NRhM10HBodiqW4IAdpbAniKzCzxiKQGEj3g93r54Sf3ZzCacMR327kHMhRLOtZPrFi8sGi/fE9/hDOX/zn7On+8xH2Hu3hSWbF4YWqmlYC3SIxejzgG65+47EOu3WIi25LynZ85CERdH4eo48Q5GFG2DDiL93339P0YF/RxxHWPOQZ9h6JJ9yvyxQsdr3X0nMkF04O/94l5BXdJQ9Hi4xuelvJnGDUV2Su8aQRjDP1F7uRWLF5IhQvIbSV1FeyIhYcNuVGPiBwjIqtF5G8iclm944Ft4w8ZSx97A79XHMcgfcvv9bBi8UIuP34O1/1uNQ+/vAFVZcq4Jq46aV9WLF7IVSftS2dbkN0ntfDtk+cypa2J5WcvGPbqecXihew6vonzPvh+NGdRWOa1bn/iLd7dOpQ9qY4L+Ybd+Of2J95CEAI+DxNbggV3P2s2hQtOhsPFtzRvI5vMIHD+78fnEcKxJFvDMd7YMFA0Ro+QHbcAinaLZRZxd29OFfNLJt0fl0zi2LAoltTsrK7M47587wvE0ok9M+jbOxAlNsz7zn2tnr4oiWTS8e/s9QjRhLMraXvTkjMXKNPam+lsC273Dz6u7pMC4iMoeVPttRuZE5pjw6n1fQXdbGZ4pW6qNRINd8cgIl7gh8CHgW7gGRH5haq+XM+48scfevojiFAwg+i/zphH31Ase5UL6St0j4cZHU20NfmLXi3N3qWNdVvDRWfU5NYN2vBeuKAkxjWnzOX2J95i6aL5TGlrwufzsHbzYNFVxRcftSed6Zkr/UMDBSeB/FlGUPzq3ucRmvwe7j7vYHr6IvQORPF7hRvPOsC1bMdL7/YzIeQnmkg6klx+jHvt0srysxdw/p0rszOucmWSQSaOf2wdYvrE5qIn3dw/plLGAzKDvsWKB3o8zlInS47Yw7GFa+Zx+Wsoql0aPJZwfy+xROUn3WoXDrSV1NWxIxYeNlxiAA4C/qaqbwKIyE+Bk4C6Jga38QevB/7joZcdJ7Rlf3qDfztujqMPPdM/vL155qkFdu4lvwvn6pOd8ePzCl6Ph3A0zhUn7JNNCgBtTR7XGT9tTR6mtG2bseJ2osqfZQSwNRxzPdkHfR7eG4ozY2KI5oCPqeObiCXUdVbSOYfOzA6kAzz99SM5/593L1iT4fcIk9qamDIOHrjwMGKJZNFFfZkptj/+f29mu+W2d9INFlkkmFsKPTPom0gmXbsC80+eE3IWC2akxnSc/9bVLg1eiz2ph0vslbCV1NVRi/In+RoxMUwD1uR83w0cnPsAEVkMLAbYbbfddlhg+Sf2ZFL58odnF/T17To+VPEK1FgC2kM+x3RVQYnl/e1Mbg1yQt4gcO6dQkbfUJJYLObYknL91kH6hvyOxVmdLYGChNQ1McTys+dz/p3bjjX5Pfh9HsdKar9PCPnJ9nFmfkdbw0OuA9WtTc4k1xIU2lsCjjUZ7S0BWoKS/b11tgUZGoq7lvcY3yTce8EheAW+ffJcAj51Ta75r/vMWxu5+ez5XJDz/m4+ez7PvLURcO4pvXEg4lgVPxhN0Bzw4vGIow23ZNrVHiLod772cHdJlQj6PEUTdqWGS+yV2BEntLGgFuVP8jVc2W0ROQ34iKqel/7+bOAgVf2C2+OrWXa7EtUeTIvHk7y9acBRXXX6xBAzJrYUzEfPLGiKJ5L4iixoGhqKu25JOaujpaDsQzQap2dgW92gzpYAPp/X8f6a/UoC6Asns49rC3nwAi1NzpkqvQNDJJJKNK4kkql9LQI+IZFQTrt522ycX1x0KEFvan/qTJsTQh4iCZjY4myzlNpG/9ga5rV/bGWPyeOyj3tjw3vM2mU8py190vHHtMv4AIORba/bHPQQS0AsnnT8ew4Nxel+L1zw79I1LkQgsO13FAp4Wb81wvl3Dj8w2Dc0xNu9hauuZ3QEaWsqf8ZPNBpnXX8kuyVsJmFPbQ0SCFR2/ReLJXh1Q3/VtjO1aq3VU43zznBltxsxMRwCXKmqH0l//3UAVf0/bo+vd2KohVJO+OUotVBcqbENxqMFiaHZFyiIsVh5iN3amx17PIwPelnfHynY6W2XtqaKTkCxWILXe/pZnHMnsOzs+bx/UgtbIwnHHxNQ8h9Yqb/HUv5oY7EE/bFYQVJq9fsr3kPaLbFXmhRy49zQH8m2Obk1OKI9rm1WUuMYbYnBB7wGHAWsBZ4BPqmqf3V7/M6YGBpdOYmr1BNBtU9A1W6vFkZDjGbnNao26lHVuIhcBPwO8AK3FEsKpj58Pk/Jq2lLLezm93sLdmsbiWq3VwujIUYzNjVcYgBQ1V8Dv653HMYYMxY15AI3Y4wx9WOJwRhjjIMlBmOMMQ6WGIwxxjg03HTVcolID/BOBU+dBGyscjjV1ugxNnp8YDFWS6PH2OjxQePF+D5V7XT7wahPDJUSkZXF5vA2ikaPsdHjA4uxWho9xkaPD0ZHjBnWlWSMMcbBEoMxxhiHsZwYltU7gBI0eoyNHh9YjNXS6DE2enwwOmIExvAYgzHGGHdj+Y7BGGOMC0sMxhhjHMZcYhCRY0RktYj8TUQuq3c8+URkuoj8QUReEZG/isgX6x1TMSLiFZHnROShesfiRkQmiMh9IvJq+vd5SL1jyiUiX07/G78kIveISPk79NSAiNwiIhtE5KWcYxNF5BEReT39ub3B4rs2/e/8oog8ICIT6hVfsRhzfvZVEVERmVSP2EoxphKDiHiBHwLHAnOAM0VkTn2jKhAHvqKqewMLgc83YIwZXwReqXcQw/ge8FtV3QvYjwaKVUSmARcDC1R1X1Il5j9R36iybgOOyTt2GfCoqs4CHk1/Xy+3URjfI8C+qjqX1H4uX9/RQeW5jcIYEZHpwIeBv+/ogMoxphIDcBDwN1V9U1WjwE+Bk+ock4OqrlPVZ9Nf95E6mU2rb1SFRKQLOA74Ub1jcSMi44APAj8GUNWoqm6pb1QFfEAovTlVM/BuneMBQFX/BGzKO3wScHv669uBj+3QoHK4xaeqD6tqPP3tU0DXDg/MGY/b7xDgu8DXgIae9TPWEsM0YE3O99004Ek3Q0RmAPsDf65vJK7+i9R/8GS9Aylid6AHuDXd3fUjEWmpd1AZqroWuI7UleM6YKuqPlzfqIY1RVXXQeriBZhc53iG8xngN/UOIp+InAisVdUX6h3L9oy1xOC2uWxDZm4RaQXuB76kqu/VO55cInI8sEFVV9U7lmH4gAOAm1R1f2CA+nZ/OKT76E8CZgK7Ai0isqi+UY1+IvINUt2xd9c7llwi0gx8A/j3esdSirGWGLqB6Tnfd9Egt++5RMRPKincrao/r3c8Lg4DThSRt0l1xx0pInfVN6QC3UC3qmbutu4jlSgaxb8Ab6lqj6rGgJ8Dh9Y5puGsF5GpAOnPG+ocTwER+RRwPHCWNt4CrT1IXQS8kP676QKeFZFd6hpVEWMtMTwDzBKRmSISIDXY94s6x+QgIkKqX/wVVf3PesfjRlW/rqpdqjqD1O/w96raUFe7qvoPYI2IzE4fOgp4uY4h5fs7sFBEmtP/5kfRQIPjLn4BfCr99aeAB+sYSwEROQa4FDhRVQfrHU8+Vf2Lqk5W1Rnpv5tu4ID0/9OGM6YSQ3pw6iLgd6T+CO9V1b/WN6oChwFnk7oKfz798dF6BzVKfQG4W0ReBOYB/7vO8WSl72TuA54F/kLqb7EhSiaIyD3Ak8BsEekWkc8CVwMfFpHXSc2qubrB4vsB0AY8kv6bWVqv+IaJcdSwkhjGGGMcxtQdgzHGmO2zxGCMMcbBEoMxxhgHSwzGGGMcLDEYY4xxsMRgzHaISH8N2pyXOw1ZRK4Uka9W+3WMqYQlBmPqYx5g61NMQ7LEYEwZROQSEXkmXff/m+ljM9L7PSxP76/wsIiE0j87MP3YJ9N7BryUXnX/LeCM9GKsM9LNzxGRx0TkTRG5uE5v0RhLDMaUSkSOBmaRKt8+D5gvIh9M/3gW8ENV3QfYApySPn4rsERVDwESkCoBTqqY2gpVnaeqK9KP3Qv4SLr9K9I1s4zZ4SwxGFO6o9Mfz5EqZbEXqYQAqYJ4z6e/XgXMSO8i1qaqT6SP/2Q77f9KVSOqupFUkbopVY3emBL56h2AMaOIAP9HVW92HEztmxHJOZQAQriXeR9Ofhv292nqwu4YjCnd74DPpPfKQESmiUjRDWtUdTPQJyIL04dyt+7sI1X0zZiGY4nBmBKld1j7CfCkiPyFVHXU7Z3cPwssE5EnSd1BbE0f/wOpwebcwWdjGoJVVzWmhkSkVVX7019fBkxV1S/WOSxjhmV9mMbU1nEi8nVSf2vvAJ+ubzjGbJ/dMRhjjHGwMQZjjDEOlhiMMcY4WGIwxhjjYInBGGOMgyUGY4wxDv8fayc/9Zrg+VEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.scatterplot(df_contracts.length,df_contracts.annual_salary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions\n",
    "\n",
    "* Try to explore the correlation of other variables in the dataset.\n",
    "* Can you think of a possible motivation for the trend we see: older apprentices with a shorter contract getting on average a higher annual salary?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sampling and uncertainty (mention)\n",
    "\n",
    "Often, we work with samples and we want the sample to be representative of the population it is taken from, in order to draw conclusions that generalise from the sample to the full population.\n",
    "\n",
    "Sampling is *tricky*. Samples have *variance* (variation between samples from the same population) and *bias* (systematic variation from the population)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 2: working with texts\n",
    "\n",
    "Let's get some basics (or a refresher) of working with texts in Python. Texts are sequences of discrete symbols (words or, more generically, tokens).\n",
    "\n",
    "Key challenge: representing text for further processing. Two mainstream approaches:\n",
    "* *Bag of words*: a text is a collection of tokens occurring with a certain frequence and assumed independently from each other within the text. The mapping from texts to features is determinsitic and straighforward, each text is represented as a vector of the size of the vocabulary.\n",
    "* *Embeddings*: a method is used (typically, neural networks), to learn a mapping from each token to a (usually small) vector representing it. A text can be represented in turn as an aggregation of these embeddings."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import the dataset\n",
    "Let us import the Elon Musk's tweets dataset in memory.\n",
    "\n",
    "<img src=\"figures/elon_loop.jpeg\" width=\"400px\" heigth=\"400px\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "root_folder = \"../data/musk_tweets\"\n",
    "df_elon = pd.read_csv(codecs.open(os.path.join(root_folder,\"elonmusk_tweets.csv\"), encoding=\"utf8\"), sep=\",\")\n",
    "df_elon['text'] = df_elon['text'].str[1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>created_at</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>849636868052275200</td>\n",
       "      <td>2017-04-05 14:56:29</td>\n",
       "      <td>'And so the robots spared humanity ... https:/...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>848988730585096192</td>\n",
       "      <td>2017-04-03 20:01:01</td>\n",
       "      <td>\"@ForIn2020 @waltmossberg @mims @defcon_5 Exac...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>848943072423497728</td>\n",
       "      <td>2017-04-03 16:59:35</td>\n",
       "      <td>'@waltmossberg @mims @defcon_5 Et tu, Walt?'</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>848935705057280001</td>\n",
       "      <td>2017-04-03 16:30:19</td>\n",
       "      <td>'Stormy weather in Shortville ...'</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>848416049573658624</td>\n",
       "      <td>2017-04-02 06:05:23</td>\n",
       "      <td>\"@DaveLeeBBC @verge Coal is dying due to nat g...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   id           created_at  \\\n",
       "0  849636868052275200  2017-04-05 14:56:29   \n",
       "1  848988730585096192  2017-04-03 20:01:01   \n",
       "2  848943072423497728  2017-04-03 16:59:35   \n",
       "3  848935705057280001  2017-04-03 16:30:19   \n",
       "4  848416049573658624  2017-04-02 06:05:23   \n",
       "\n",
       "                                                text  \n",
       "0  'And so the robots spared humanity ... https:/...  \n",
       "1  \"@ForIn2020 @waltmossberg @mims @defcon_5 Exac...  \n",
       "2       '@waltmossberg @mims @defcon_5 Et tu, Walt?'  \n",
       "3                 'Stormy weather in Shortville ...'  \n",
       "4  \"@DaveLeeBBC @verge Coal is dying due to nat g...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_elon.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2819, 3)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_elon.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Natural Language Processing in Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import some of the most popular libraries for NLP in Python\n",
    "import spacy\n",
    "import nltk\n",
    "import string\n",
    "import sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A typical NLP pipeline might look like the following:\n",
    "    \n",
    "<img src=\"figures/spacy_pipeline.png\" width=\"600px\" heigth=\"600px\">\n",
    "\n",
    "### Tokenization: splitting a text into constituent tokens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.tokenize import TweetTokenizer, word_tokenize\n",
    "tknzr = TweetTokenizer(preserve_case=True, reduce_len=False, strip_handles=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"@ForIn2020 @waltmossberg @mims @defcon_5 Exactly. Tesla is absurdly overvalued if based on the past, but that's irr\\xe2\\x80\\xa6 https://t.co/qQcTqkzgMl\"\n"
     ]
    }
   ],
   "source": [
    "example_tweet = df_elon.text[1]\n",
    "print(example_tweet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\"', '@ForIn2020', '@waltmossberg', '@mims', '@defcon_5', 'Exactly', '.', 'Tesla', 'is', 'absurdly', 'overvalued', 'if', 'based', 'on', 'the', 'past', ',', 'but', \"that's\", 'irr', '\\\\', 'xe2', '\\\\', 'x80', '\\\\', 'xa6', 'https://t.co/qQcTqkzgMl', '\"']\n",
      "['``', '@', 'ForIn2020', '@', 'waltmossberg', '@', 'mims', '@', 'defcon_5', 'Exactly', '.', 'Tesla', 'is', 'absurdly', 'overvalued', 'if', 'based', 'on', 'the', 'past', ',', 'but', 'that', \"'s\", 'irr\\\\xe2\\\\x80\\\\xa6', 'https', ':', '//t.co/qQcTqkzgMl', \"''\"]\n"
     ]
    }
   ],
   "source": [
    "tkz1 = tknzr.tokenize(example_tweet)\n",
    "print(tkz1)\n",
    "tkz2 = word_tokenize(example_tweet)\n",
    "print(tkz2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Question: can you spot what the Twitter tokenizer is doing instead of a standard one?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'!\"#$%&\\'()*+,-./:;<=>?@[\\\\]^_`{|}~'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "string.punctuation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# some more pre-processing\n",
    "\n",
    "def filter(tweet):\n",
    "    \n",
    "    # remove punctuation and short words and urls\n",
    "    tweet = [t for t in tweet if t not in string.punctuation and len(t) > 3 and not t.startswith(\"http\")]\n",
    "    return tweet\n",
    "\n",
    "def tokenize_and_string(tweet):\n",
    "    \n",
    "    tkz = tknzr.tokenize(tweet)\n",
    "    \n",
    "    tkz = filter(tkz)\n",
    "    \n",
    "    return \" \".join(tkz)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\"', '@ForIn2020', '@waltmossberg', '@mims', '@defcon_5', 'Exactly', '.', 'Tesla', 'is', 'absurdly', 'overvalued', 'if', 'based', 'on', 'the', 'past', ',', 'but', \"that's\", 'irr', '\\\\', 'xe2', '\\\\', 'x80', '\\\\', 'xa6', 'https://t.co/qQcTqkzgMl', '\"']\n",
      "['@ForIn2020', '@waltmossberg', '@mims', '@defcon_5', 'Exactly', 'Tesla', 'absurdly', 'overvalued', 'based', 'past', \"that's\"]\n"
     ]
    }
   ],
   "source": [
    "print(tkz1)\n",
    "print(filter(tkz1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_elon[\"clean_text\"] = df_elon[\"text\"].apply(tokenize_and_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>created_at</th>\n",
       "      <th>text</th>\n",
       "      <th>clean_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>849636868052275200</td>\n",
       "      <td>2017-04-05 14:56:29</td>\n",
       "      <td>'And so the robots spared humanity ... https:/...</td>\n",
       "      <td>robots spared humanity</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>848988730585096192</td>\n",
       "      <td>2017-04-03 20:01:01</td>\n",
       "      <td>\"@ForIn2020 @waltmossberg @mims @defcon_5 Exac...</td>\n",
       "      <td>@ForIn2020 @waltmossberg @mims @defcon_5 Exact...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>848943072423497728</td>\n",
       "      <td>2017-04-03 16:59:35</td>\n",
       "      <td>'@waltmossberg @mims @defcon_5 Et tu, Walt?'</td>\n",
       "      <td>@waltmossberg @mims @defcon_5 Walt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>848935705057280001</td>\n",
       "      <td>2017-04-03 16:30:19</td>\n",
       "      <td>'Stormy weather in Shortville ...'</td>\n",
       "      <td>Stormy weather Shortville</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>848416049573658624</td>\n",
       "      <td>2017-04-02 06:05:23</td>\n",
       "      <td>\"@DaveLeeBBC @verge Coal is dying due to nat g...</td>\n",
       "      <td>@DaveLeeBBC @verge Coal dying fracking It's ba...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   id           created_at  \\\n",
       "0  849636868052275200  2017-04-05 14:56:29   \n",
       "1  848988730585096192  2017-04-03 20:01:01   \n",
       "2  848943072423497728  2017-04-03 16:59:35   \n",
       "3  848935705057280001  2017-04-03 16:30:19   \n",
       "4  848416049573658624  2017-04-02 06:05:23   \n",
       "\n",
       "                                                text  \\\n",
       "0  'And so the robots spared humanity ... https:/...   \n",
       "1  \"@ForIn2020 @waltmossberg @mims @defcon_5 Exac...   \n",
       "2       '@waltmossberg @mims @defcon_5 Et tu, Walt?'   \n",
       "3                 'Stormy weather in Shortville ...'   \n",
       "4  \"@DaveLeeBBC @verge Coal is dying due to nat g...   \n",
       "\n",
       "                                          clean_text  \n",
       "0                             robots spared humanity  \n",
       "1  @ForIn2020 @waltmossberg @mims @defcon_5 Exact...  \n",
       "2                 @waltmossberg @mims @defcon_5 Walt  \n",
       "3                          Stormy weather Shortville  \n",
       "4  @DaveLeeBBC @verge Coal dying fracking It's ba...  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_elon.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save cleaned up version\n",
    "\n",
    "df_elon.to_csv(os.path.join(root_folder,\"df_elon.csv\"), index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building a dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2819, 7864)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "count_vect = CountVectorizer(lowercase=False, tokenizer=tknzr.tokenize)\n",
    "X_count = count_vect.fit_transform(df_elon.clean_text)\n",
    "X_count.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6617"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_list = count_vect.get_feature_names()  \n",
    "count_list = X_count.toarray().sum(axis=0)\n",
    "dictionary = dict(zip(word_list,count_list))\n",
    "count_vect.vocabulary_.get(\"robots\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_count[:,count_vect.vocabulary_.get(\"robots\")].toarray().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary[\"robots\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions\n",
    "\n",
    "* Find the tokens most used by Elon.\n",
    "* Find the twitter users most referred to by Elon (hint: use the @ handler to spot them)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Tesla', 322),\n",
       " ('Model', 236),\n",
       " ('that', 223),\n",
       " ('will', 218),\n",
       " ('with', 177),\n",
       " ('@SpaceX', 169),\n",
       " ('from', 163),\n",
       " ('this', 159),\n",
       " ('@TeslaMotors', 149),\n",
       " ('launch', 124)]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary_list = sorted(dictionary.items(), key=lambda x:x[1], reverse=True)\n",
    "[d for d in dictionary_list][:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('@SpaceX', 169),\n",
       " ('@TeslaMotors', 149),\n",
       " ('@elonmusk', 85),\n",
       " ('@NASA', 48),\n",
       " ('@Space_Station', 19),\n",
       " ('@FredericLambert', 17),\n",
       " ('@ID_AA_Carmack', 15),\n",
       " ('@WIRED', 14),\n",
       " ('@vicentes', 14),\n",
       " ('@BadAstronomer', 11)]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary_list_users = sorted(dictionary.items(), key=lambda x:x[1], reverse=True)\n",
    "[d for d in dictionary_list if d[0].startswith('@')][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Representing tweets as vectors\n",
    "\n",
    "Texts are of variable length and need to be represented numerically in some way. Most typically, we represent them as *equally-sized vectors*.\n",
    "\n",
    "Actually, this is what we have already done! Let's take a closer look at `X_count` above.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id                                           849636868052275200\n",
       "created_at                                  2017-04-05 14:56:29\n",
       "text          'And so the robots spared humanity ... https:/...\n",
       "clean_text                               robots spared humanity\n",
       "Name: 0, dtype: object"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This is the first Tweet of the data frame\n",
    "\n",
    "df_elon.loc[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's get the vector representation for this Tweet\n",
    "\n",
    "vector_representation = X_count[0,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# there are 3 positions not to zero, as we would expect: the vector contains 1 in the columns related to the 3 words that make up the Tweet. \n",
    "# It would contain a number higher than 1 if a given word were occurring multiple times.\n",
    "\n",
    "np.sum(vector_representation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "1\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# Let's check that indeed the vector contains 1s for the right words\n",
    "# Remember, the vector has shape (1 x size of the vocabulary)\n",
    "\n",
    "print(vector_representation[0,count_vect.vocabulary_.get(\"robots\")])\n",
    "print(vector_representation[0,count_vect.vocabulary_.get(\"spared\")])\n",
    "print(vector_representation[0,count_vect.vocabulary_.get(\"humanity\")])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Term Frequency - Inverse Document Frequency\n",
    "We can use boolean counts (1/0) and raw counts (as we did before) to represent a Tweet over the space of the vocabulary, but there exist improvements on this basic idea. For example, the TF-IDF weighting scheme:\n",
    "\n",
    "$tfidf(t, d, D) = tf(t, d) \\cdot idf(t, D)$\n",
    "\n",
    "$tf(t, d) = f_{t,d}$\n",
    "\n",
    "$idf(t, D) = log \\Big( \\frac{|D|}{|{d \\in D: t \\in d}|} \\Big)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2819, 7864)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "count_vect = TfidfVectorizer(lowercase=False, tokenizer=tknzr.tokenize)\n",
    "X_count_tfidf = count_vect.fit_transform(df_elon.clean_text)\n",
    "X_count_tfidf.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.7226760995112569"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_count_tfidf[0,:].sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_count[0,:].sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sparse vectors (mention)\n",
    "How is Python representing these vectors in memory? Most of their cells are set to zero. \n",
    "\n",
    "We call any vector or matrix whose cells are mostly to zero *sparse*.\n",
    "There are efficient ways to store them in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<1x7864 sparse matrix of type '<class 'numpy.float64'>'\n",
       "\twith 3 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_count_tfidf[0,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Spacy pipelines\n",
    "\n",
    "Useful to construct sequences of pre-processing steps: https://spacy.io/usage/processing-pipelines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install the required model\n",
    "\n",
    "#!python -m spacy download en_core_web_sm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# without this line spacy is not able to find the downloaded model\n",
    "\n",
    "#!python -m spacy link --force en_core_web_sm en_core_web_sm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load a pre-trained pipeline (Web Small): https://spacy.io/usage/models\n",
    "\n",
    "#!python -m spacy download en_core_web_sm\n",
    "nlp = spacy.load('en_core_web_sm')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*.. the model’s meta.json tells spaCy to use the language \"en\" and the pipeline [\"tagger\", \"parser\", \"ner\"]. spaCy will then initialize spacy.lang.en.English, and create each pipeline component and add it to the processing pipeline. It’ll then load in the model’s data from its data directory and return the modified Language class for you to use as the nlp object.*\n",
    "\n",
    "Let's create a simple pipeline that does **lemmatization**, **part of speech tagging** and **named entity recognition** using spaCy models.\n",
    "\n",
    "*If you don't know what these NLP tasks are, please ask!*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "tweet_pos = list()\n",
    "tweet_ner = list()\n",
    "tweet_lemmas = list()\n",
    "\n",
    "for tweet in df_elon.text.values:\n",
    "    spacy_tweet = nlp(tweet)\n",
    "    \n",
    "    local_tweet_pos = list()\n",
    "    local_tweet_ner = list()\n",
    "    local_tweet_lemmas = list()\n",
    "    \n",
    "    for sentence in list(spacy_tweet.sents):\n",
    "        # --- lemmatization, remove punctuation and stop wors\n",
    "        local_tweet_lemmas.extend([token.lemma_ for token in sentence if not token.is_punct | token.is_stop])\n",
    "        local_tweet_pos.extend([token.pos_ for token in sentence if not token.is_punct | token.is_stop])\n",
    "    for ent in spacy_tweet.ents:\n",
    "        local_tweet_ner.append(ent)\n",
    "\n",
    "    tweet_pos.append(local_tweet_pos)\n",
    "    tweet_ner.append(local_tweet_ner)\n",
    "    tweet_lemmas.append(local_tweet_lemmas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['and', 'robot', 'spar', 'humanity', 'https://t.co/v7jujqwfcv']"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet_lemmas[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['CCONJ', 'NOUN', 'VERB', 'NOUN', 'X']"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet_pos[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweet_ner[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Shortville]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# but it actually works!\n",
    "\n",
    "tweet_ner[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Note: we are really just scratching the surface of spaCy, but it is worth knowing it's there.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Searching tweets\n",
    "\n",
    "Once we have represented Tweets as vectors, we can easily find similar ones using basic operations such as filtering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "robots spared humanity\n"
     ]
    }
   ],
   "source": [
    "target = 0\n",
    "print(df_elon.clean_text[target])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "condition = X_count_tfidf[target,:] > 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 5198)\tTrue\n",
      "  (0, 6617)\tTrue\n",
      "  (0, 6949)\tTrue\n"
     ]
    }
   ],
   "source": [
    "print(condition)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_filtered = X_count_tfidf[:,np.ravel(condition.toarray())]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<2819x3 sparse matrix of type '<class 'numpy.float64'>'\n",
       "\twith 16 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_filtered"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 0)\t0.49528340735923404\n",
      "  (0, 2)\t0.6406029997190413\n",
      "  (0, 1)\t0.5867896924329815\n",
      "  (217, 0)\t0.2972381925908634\n",
      "  (271, 0)\t0.3284547085372313\n",
      "  (464, 0)\t0.22738802397468952\n",
      "  (473, 0)\t0.5667220639589731\n",
      "  (734, 1)\t0.3846355279044392\n",
      "  (940, 0)\t0.27312597149485407\n",
      "  (1004, 0)\t0.28161575586607157\n",
      "  (1550, 1)\t0.33303254164524276\n",
      "  (1862, 0)\t0.3196675199194523\n",
      "  (2493, 0)\t0.2685018991334563\n",
      "  (2559, 0)\t0.311452470142279\n",
      "  (2565, 0)\t0.2645117238497897\n",
      "  (2661, 0)\t0.2729016388865858\n"
     ]
    }
   ],
   "source": [
    "print(X_filtered)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([   0,  217,  271,  464,  473,  940, 1004, 1862, 2493, 2559, 2565,\n",
       "        2661,    0,  734, 1550,    0], dtype=int32),\n",
       " array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2], dtype=int32),\n",
       " array([0.49528341, 0.29723819, 0.32845471, 0.22738802, 0.56672206,\n",
       "        0.27312597, 0.28161576, 0.31966752, 0.2685019 , 0.31145247,\n",
       "        0.26451172, 0.27290164, 0.58678969, 0.38463553, 0.33303254,\n",
       "        0.640603  ]))"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy import sparse\n",
    "\n",
    "sparse.find(X_filtered)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "tweet_indices = list(sparse.find(X_filtered)[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TARGET: robots spared humanity\n",
      "1)@JustBe74 important make humanity proud this case particular duty owed American taxpayer\n",
      "2)@pud Faith restored humanity French toast money\n",
      "3)humanity have exciting inspiring future cannot confined Earth forever @love_to_dream #APSpaceChat\n",
      "4)@ShireeshAgrawal like humanity\n",
      "5)Creating neural lace thing that really matters humanity achieve symbiosis with machines\n",
      "6)@tzepr Certainly agree that first foremost triumph humanity cheering good spirit\n",
      "7)@ReesAndersen @FLIxrisk believe that critical ensure good future humanity\n",
      "8)@NASA #Mars hard x99s worth risks extend humanity x99s frontier beyond Earth Learn about neighbor planet\n",
      "9)Astronomer Royal Martin Rees soon will robots take over world @Telegraph\n",
      "10)@thelogicbox @IanrossWins Mars critical long-term survival humanity life Earth know\n",
      "11)humanity wishes become multi-planet species then must figure move millions people Mars\n",
      "12)Sure feels weird find myself defending robots\n",
      "13)Neil Armstrong hero humanity spirit will carry stars\n"
     ]
    }
   ],
   "source": [
    "print(\"TARGET: \" + df_elon.clean_text[target])\n",
    "\n",
    "for n, tweet_index in enumerate(list(set(tweet_indices))):\n",
    "    if tweet_index != target:\n",
    "        print(str(n) +\")\"+ df_elon.clean_text[tweet_index])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Questions\n",
    "\n",
    "* Can you rank the matched tweets using their tf-idf weights, so to put higher weighted tweets first?\n",
    "* Which limitations do you think a bag of words representation has?\n",
    "* Can you spot any limitations of this approach based on similarity measures over bag of words representations?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}